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Preface 


This book is for students and scholars who want to think 
about the big questions: Are we rational? What at all 
does it mean to be rational? Why do we sometimes devi- 
ate from the norms of rationality? What happens in our 
brain when we reason and decide? What is a good or bad 
decision? How do we come to sensible beliefs and knowl- 
edge? These are questions that people have wondered 
about since the beginning of humankind. And these are 
questions about which, since ancient times, thinkers have 
had much to say. The aim of The Handbook of Rational- 
ity is to show how (analytic) philosophers and (cognitive) 
psychologists think about these big questions. In the past 
decades, both disciplines have made tremendous advances 
in better understanding the very nature of human ratio- 
nality. Yet, the interaction between the fields is developed 
only in rudimentary forms. The consequence is that no 
publications exist that provide students and scholars with 
an easily accessible integrated overview about the state of 
the art in the psychology and philosophy of rationality. 
The Handbook of Rationality seeks to fill this gap and to be 
a novel resource for students and scholars in psychology 
and philosophy, but also in neighboring areas, such as 
economics, neuroscience, artificial intelligence, linguistics, 
law, sociology, anthropology, or education. 

This handbook has not become so voluminous because 
we wanted it to be. A little less work with the publica- 
tion of this volume would have been all right. But a less 
complete treatment would not have done justice to the 
importance of the subject. Of course we were fully aware 
of the complexity and breadth of the topic of rationality 
when we started to think about the chapters and authors 
that we wanted to collect in this handbook. However, 
during our work, we repeatedly encountered new topics 
and issues that we had not considered at the beginning 
because they only arose from our interdisciplinary coop- 
eration. If we had merely concentrated on our respec- 
tive home disciplines, we would have missed many of 
these interesting questions lying in the no man’s land 
between the disciplines. 


What has greatly facilitated our interdisciplinary col- 
laboration and made it so productive is the fact that we 
already collaborated for many years within the Priority 
Program SPP 1516, New Frameworks of Rationality, which 
has been funded by the German Research Foundation 
(Deutsche Forschungsgemeinschaft [DFG]) from 2011 
to 2018. The program consisted of 15 research projects 
from psychology, philosophy, and artificial intelligence 
and often combined methods from these disciplines 
within highly interdisciplinary projects. The program was 
headed by Markus Knauff and coinitiated by Wolfgang 
Spohn, the cognitive psychologists Ralph Hertwig and 
Michael Waldmann, and the philosopher Gerhard 
Schurz. Later, the computer scientist Gabriele Kern-Isberner 
joined this governing board. In this priority program, we 
gathered many times, worked together very intensively, 
worked on many joint publications, and spent many, 
many hours of intensive discussion (www.spp1516.de). 
All this resulted in enormous shared benefit and progress 
in mutual understanding. 

At some point, we thought we should share our pro- 
gress with a broader scientific public, and so the idea 
for this handbook was born in the summer of 2015. We 
are very grateful to the DFG for the generous funding 
of the interdisciplinary research program. We are aware 
that Germany is one of the few countries that still invest 
so much in basic research without expecting results that 
are directly economically exploitable. This is a privilege 
we greatly appreciate. Many scientists from this pro- 
gram have also contributed to this handbook, although 
we have taken great care to include many other experts 
from the worldwide community of rationality research. 
The handbook is a truly international enterprise. We are 
very pleased that most of the world’s leading experts 
were very enthusiastic and immediately agreed to con- 
tribute a chapter to this handbook. The different disci- 
plines use quite different formats for footnotes, indexing, 
references, and other stylistic matters. To establish uni- 
formity throughout the handbook, we have chosen to 


use the rules from the Publication Manual of the American 
Psychological Association (6th edition, 2010). 

Our list of acknowledgments is long. First, we thank all 
our authors. In order to cover our field comprehensively, 
we had to engage the best authors across the disciplines. 
We expected that many authors would deliver too late, 
that we would have to wait a very long time for many 
contributions, that we would have to send out dozens of 
reminders, and so on. But we were too pessimistic. Most 
of our authors were very reliable, delivered their contribu- 
tions fairly punctually, and were also very cooperative and 
responsive. We are very sad that one of our initial authors, 
Sieghard Beller, passed away before he could deliver 
his chapter and that another author, Arthur Merin, died 
shortly after finishing his chapter. We are most grateful to 
all of the authors for the exemplary cooperation. 

We are also indebted to all our reviewers. In general, 
each chapter was reviewed by two reviewers, one from 
psychology and one from philosophy. In most cases, this 
was possible, and sometimes we even received three or 
four reviews from colleagues with different disciplinary 
backgrounds. For each chapter, at least one review came 
from an author of another chapter and the other from 
an external reviewer. We are very thankful to the follow- 
ing external reviewers: 


Robert Audi 
Anthony Blair 


Simon Handley 

Sven Ove Hansson 
Thomas Blanchard Tim Henning 
Luc Bovens 


Matteo Colombo 


Risto Hilpinen 
Franz Huber 
Carsten Koenneker 
Jakob Koscholke 


Annette Lessm6llmann 


Vincenzo Crupi 
David Danks 


Lorraine Daston 


Eddie Dekel 


Emmanuelle-Anna 
Dietz Saldanha 


Igor Douven 
Frank Fischer 
Christian Freksa 
Peter Gardenfors 
Valeria Giardino 
Itzhak Gilboa 
Anthony Gillies 
Herbert Gintis 
Geoffrey Goodwin 
Werner Gtith 
Jaap Hage 


Tania Lombrozo 
Hugo Mercier 
Philippe Mongin 
Martin Monti 
Josef Perner 
Martin Peterson 
Eric Raidl 
Christian Ruff 
Giovanni Sartor 
Walter Schaeken 
Mark Schroeder 
Armin Schulz 
Keith Stenning 
Bob Sternberg 
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Jakub Szymanik Peter Wakker 
Raimo Tuomela Paul Weirich 
Gottfried Vosgerau Gregory Wheeler 


Thomas Voss Kevin Zollman 


We are very sad that Christian Freksa died in an acci- 
dent before the book came out. 

We mentioned that we could build on a long previ- 
ous cooperation within the Priority Program New Frame- 
works of Rationality. Here we want to thank all members 
of the program for the lively and productive coopera- 
tion. It was most instructive. 

We are deeply grateful to Christopher von Btilow, who 
checked all the manuscripts and brought them into the 
final uniform shape. He did so with unsurpassable care, 
at times at which we editors were already exhausted. It is 
also his merit that this project came to a good end. 

We also thank Lupita Estefania Gazzo Castafieda, 
Andreas Kemmerling, and Christoph Klauer for their com- 
ments on an earlier version of the introductory chapter. 
Knauff wrote parts of the long introductory chapter at the 
University of California, Santa Barbara (UCSB), and thus 
wants to thank Daniel Montello for his hospitality and 
the perfect environment for focused thinking and writing. 
Knauff also wants to thank Estefania and his whole team 
for the many interesting and lively discussions. Spohn 
is doubly grateful to the DFG for support: in 2016-2018 
through the research unit FOR 1614 at the University of 
Konstanz and in 2019-2020 through the excellence clus- 
ter EXC 2064/1 (project number 390727645) at the Uni- 
versity of Tiibingen. 

We are also most grateful to Philip Laughlin and the 
team from MIT Press. The interaction with all of them 
was always pleasant and professional. This handbook was 
a huge enterprise and thus, in the final phase, an enor- 
mous work for the publisher, which was performed in an 
excellent way. 

Normally, the editors form a unit that cannot thank 
itself. Still, we feel the urge to do this. We can only thank 
each other for the untiring willingness to work together 
over such a long period of time on such a large project. It 
was a great scientific experience that neither of us wants 
to miss, even though we spent so many days and nights 
discussing and Skyping, as well as exchanging (about 
3,000) emails. Our spouses, Erika and Ulli, have faithfully 
accompanied our work with their understanding, charity, 
and support. We dedicate our work to them! 


Markus Knauff and Wolfgang Spohn 


Gief$en and Konstanz, January 2021 


Overview of the Handbook 


Markus Knauff and Wolfgang Spohn 


This handbook seeks to cover its topic, rationality, in 
an interdisciplinary way, more than any other exist- 
ing handbook on this topic. Surely it cannot fulfill this 
intention completely. The topic is just too large. But it 
intends to reach broad representativity at least in its two 
main disciplines, philosophy and psychology. In addi- 
tion, it presents some insights from disciplines such as 
artificial intelligence, behavioral and microeconomics, 
and neuroscience, although these areas are only par- 
tially covered here. The current program is already very 
ambitious. 

The handbook is not organized along disciplinary 
lines of psychology and philosophy. This would be in 
clear contradiction to our interdisciplinary endeavor. 
Rather, it is organized along the common themes that 
occur in both fields, albeit often in different forms and 
under different names. In our introductory chapter, we 
propose various distinctions for classifying the over- 
whelming research on rationality. Certainly, the most 
important one is that between theoretical and practi- 
cal rationality. This distinction is mirrored in the head- 
ings of part II and part III, the two main parts of this 
handbook. These parts are preceded by part I, which 
addresses some fundamental or propaedeutic matters 
concerning the history, some paradigms, and some key 
issues of research on rationality. The handbook con- 
cludes with part IV, which presents further facets of 
rationality that are relevant for both theoretical and 
practical rationality. 

Each of the four parts consists of several sections, 
which in turn each contain several chapters, so that we 
ended up with 15 sections and 65 chapters. The order of 
chapters within a section does not have a particular sig- 
nificance. We just had to find one that creates connec- 
tions between the different disciplinary backgrounds of 
the authors. The assignment of chapters to sections was 
more difficult. The field is so variegated and interlinked 
that no classification can be perfect. Still, we hope that 
we have organized the handbook in a way that makes it 
easy to read. 


In our introductory chapter, we systematize the field 
by means of four binary distinctions: theoretical versus 
practical rationality, normative versus descriptive theo- 
ries of rationality, individual versus social rationality, 
and outcome- versus process-oriented accounts of ratio- 
nality. Since the distinctions can be combined with each 
other, we end up with a system of 16 compartments. Our 
chapter says something about most of the 16 compart- 
ments and uses this system not only to give guidance to 
the readers of this handbook but also to suggest a new 
classification system for researchers in human rational- 
ity. We recommend all readers to study this introductory 
chapter. This will certainly make it easier to orient one- 
self in, and to contextualize, the rich material collected 
in this handbook. 


Part I: Origins and Key Issues of Rationality 


The chapters in section 1 deal with the history of ratio- 
nality research and the cortical and evolutionary foun- 
dations of rational thinking. In chapter 1.1, Thomas 
Sturm describes the history of philosophical conceptions 
of rationality, which were just as much psychological 
conceptions. Since this history is so enormously rich, he 
just focuses on the emergence of the distinction between 
a descriptive and a normative perspective, which is so 
important for the entire handbook. This philosophical 
chapter is then complemented by chapter 1.2, in which 
Jonathan Evans summarizes the younger history of the 
psychology of reasoning from his personal point of view. 
Then, in chapter 1.3, Gerhard Schurz looks even farther 
back in time by speculating about the evolution of ratio- 
nality. In chapter 1.4, Vinod Goel deals with the material 
basis of rationality, the brain. He presents findings from 
cognitive neuroscience and brain imaging on which cor- 
tical networks are involved in different kinds of rational 
thinking. 

The chapters in section 2 are concerned with some 
substantial philosophical and psychological topics of 
rationality. The section begins with chapter 2.1, in which 
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John Broome deals with the relation between rationality 
and reasoning. Both keywords are extensively used in 
this handbook, but they do not seem to denote exactly 
the same matter. The terms are also used quite differ- 
ently in philosophy and psychology. In chapter 2.2, 
Ralph Wedgwood discusses the distinction between the- 
oretical and practical rationality. This is also the top- 
level distinction in this handbook. The next chapters 
present various cognitive approaches to rationality. In 
chapter 2.3, Philip Johnson-Laird gives an overview of the 
theory of mental models, which is a far-reaching frame- 
work for explaining accurate and fallacious human rea- 
soning. Another cognitive framework is the heuristics 
and biases approach, which is presented and critically 
discussed in chapter 2.4 by Klaus Fiedler, Johannes Prager, 
and Linda McCaughey. An equally influential framework 
is established by dual-process theories of reasoning. This 
framework and the related empirical evidence are pre- 
sented by Karl Christoph Klauer in chapter 2.5. We could 
have placed the Bayesian reasoning theories here, too, 
as these theories also have a quite universal ambition. 
However, the relevant chapter by Chater and Oaksford 
fits even better in the section that is particularly dedi- 
cated to probabilistic reasoning. The last chapter in sec- 
tion 2 is a big jump toward artificial intelligence. In their 
chapter 2.6, Johan van Benthem, Fenrong Liu, and Sonja 
Smets outline the logico-computational perspective on 
rationality. They show how this approach can be effi- 
ciently used to solve rationality problems in computers. 
This chapter also creates a bridge to section 3 in part II, 
which is about logical and deductive reasoning. 


Part Il: Theoretical Rationality 


Sections 3-7 are concerned with the key issues of theo- 
retical (or “epistemic” or “doxastic”) rationality and rea- 
soning. The first term originates from philosophy and is, 
in fact, not as common in psychology, in which cogni- 
tive theories of reasoning are a very active research field. 
However, the philosophical term is much broader and 
thus serves as the heading for the following sections. 
Not surprisingly, this part of the handbook starts with 
the two dominating reasoning paradigms: deductive 
logic as already conceived in ancient philosophy and 
probabilistic thinking as developed since the middle of 
the 17th century. These two paradigms are represented 
in sections 3 and 4, respectively. 

The chapters in section 3 are concerned with deduc- 
tive logic and reasoning. Of course, the principles of 
propositional and first-order logic have long been part 
of the basic knowledge of our disciplines. Yet we wanted 
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to explain at least in one place of this handbook what 
a logical proof is and how it is related to rational belief. 
This is done in chapter 3.1 by Florian Steinberger, who 
added some novel ideas to this classical topic. Proposi- 
tional and first-order logic are also important in cogni- 
tive research on human reasoning. In chapter 3.2, David 
O’Brien presents the natural-logic account and some evi- 
dence in support of this account. In chapter 3.3, Sangeet 
Khemlani explains the cognitive foundations of syllogis- 
tic reasoning, which is a fragment of first-order logic. 
His approach is based on mental models as described in 
chapter 2.3. 

Then, section 4 is concerned with approaches from 
probability theory. In chapter 4.1, Alan Hajek and Julia 
Staffel explain the normative foundations of Bayesian- 
ism. Chapter 4.2 by Stephan Hartmann complements this 
by explaining the usefulness of the theory of so-called 
Bayes nets. A further complement is given in chapter 4.3 
by Arthur Merin, which unfolds the probabilistic core 
of all considerations of relevance. This is important for 
accounts of reasoning, since premises or arguments are 
usually assumed to be relevant for their conclusions. In 
chapter 4.4, Niki Pfeifer presents his probability logic. 
In chapter 4.5, Nick Chater and Mike Oaksford defend their 
psychological perspective on rationality as conceived 
within Bayesianism. This, in turn, is complemented by 
chapter 4.6 by Klaus Oberauer and Danielle Pessach, who 
explain what, for (some) psychologists, conditionals have 
to do with probability. Chapter 4.7 by Didier Dubois and 
Henri Prade also proceeds from the assumption that beliefs, 
or epistemic states in general, come in degrees. This is why 
the chapter is placed in this section. Yet, in their account, 
rational degrees of belief do not behave like probabilities 
but in certain other ways. We think it is important to 
acknowledge that such alternatives exist. 

Logic and probability theory are, of course, not the 
only paradigms of theoretical rationality and reasoning. 
In fact, many other frameworks exist, and many of them 
are not restricted to quantitative conceptions of degrees 
of belief, as in Bayesianism. These alternative accounts 
are represented in the remaining sections of part II. 

The chapters in section 5 are concerned with qualita- 
tive representations of belief and the related accounts of 
reasoning. In chapter 5.1, Hans van Ditmarsch presents 
the basic theory of doxastic and epistemic logic. Then, 
belief revision theory is concerned with the rational 
change of epistemic states. This normative account of 
belief revision is described in chapter 5.2 by Hans Rott. 
The dynamic account of belief revision theory is com- 
pleted by ranking theory. This theory is laid out in chap- 
ter 5.3 by Gabriele Kern-Isberner, Niels Skovgaard-Olsen, 
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and Wolfgang Spohn. The dynamics of belief is also impor- 
tant in cognitive theories of defeasible reasoning. In 
chapter 5.4, Lupita Estefania Gazzo Castafieda and Markus 
Knauff describe the empirical results on human belief 
revision and defeasible reasoning. They also explain how 
qualitative and quantitative theories try to account for 
the empirical findings. Defeasible reasoning, finally, is 
closely related to argumentation, where (rational) argu- 
ments drive (rational) epistemic change. The psycho- 
logical perspective on argumentation is presented in 
chapter 5.5 by Ulrike Hahn and Peter Collins. Chapter 5.6 
by John Woods deals with the same matter from a philo- 
sophical perspective. 

There is also a close relation between epistemic change, 
on the one hand, and conditional and counterfactual rea- 
soning, on the other, although the relation is not easy 
to specify. The broad range of approaches to reasoning 
with conditionals and counterfactuals is the topic of sec- 
tion 6. It begins with chapter 6.1 by William Starr, who 
reviews the different attempts to capture the logic of con- 
ditional and counterfactual constructions. In psychology, 
so-called supposition theory has become prominent. This 
attempt to explain human conditional reasoning is rep- 
resented in chapter 6.2 by David Over and Nicole Cruz. In 
chapter 6.3, Ruth Byrne and Orlando Espino explain how 
human reasoners deal with counterfactual inferences. A 
special, but psychologically particularly relevant, case of 
conditional reasoning, so-called utility conditionals, is 
treated by Jean-Francois Bonnefon in chapter 6.4. 

From counterfactual reasoning, it is just a small step 
to causal reasoning and its subform, diagnostic reason- 
ing. The importance of causal reasoning among our 
many reasoning activities cannot be overemphasized. 
This is a huge topic on its own. Thus, we devote the 
complete section 7 to it. In chapter 7.1, Judea Pearl pre- 
sents his influential account of causal (and counterfac- 
tual) inference in probabilistic or statistical terms. How 
this kind of reasoning actually works in humans is dis- 
cussed by Michael Waldmann in chapter 7.2. The special 
case of diagnostic reasoning is dealt with by Bjérn Meder 
and Ralf Mayrhofer in chapter 7.3. This is what this hand- 
book offers concerning theoretical or epistemic rational- 
ity and reasoning. 


Part Ill: Practical Rationality 


Sections 8-12 deal with the core topics of practical 
rationality and decision making. Again, the first term is 
familiar in philosophy, and the second is mostly used 
in psychology. The terms are not equivalent, but they 
are close enough to be treated under one heading. We 
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think that in this way, we can best highlight the inter- 
disciplinary links and show how both research areas are 
connected. 

This part of the handbook should of course start 
with accounts of individual decision making, which is 
the topic of section 8. In chapter 8.1, Till Griine-Yanoff 
explains the main ideas of preference and utility theory. 
Next, chapter 8.2 by Martin Peterson presents standard 
decision theory and the normative arguments in its 
favor. However, this account has been criticized from the 
psychological as well as from the economic side. Indeed, 
prospect theory arose from the observation that people 
often deviate from expected utility maximization. This 
theory is explained by Andreas Gléckner in chapter 8.3. 
However, there are still further ways how beliefs and 
desires can be combined to determine a rational deci- 
sion. Some of them are presented in chapter 8.4 by Brian 
Hill. Another critical view on economic decision theory 
comes from the theory of bounded rationality, which is 
influential in psychology and economics. This approach 
is reviewed in chapter 8.5 by Ralph Hertwig and Anastasia 
Kozyreva. The section ends with a chapter 8.6 by Valerie 
Thompson, Shira Elqayam, and Rakefet Ackerman, which 
is concerned with the connection between rationality 
and metacognition. The authors report empirical results 
that show that metacognitive control and monitoring 
processes are important for promoting our rational per- 
formance. Thinking about our own thinking makes us 
more rational. This chapter is quite general, and it might 
have fit under other headlines, but it also connects well 
to the previous chapters on practical rationality. 

Then, section 9 deals with game theory, the other 
standard theory of practical rationality. We treat here 
only so-called noncooperative game theory, which is, 
strictly speaking, still about individual practical ratio- 
nality but in a social context. Chapter 9.1 by Max Albert 
and Hartmut Kliemt presents the classical theory, as it has 
been developed since the 1940s. In the 1980s, epistemic 
game theory emerged, which promises a more rigorous 
rationalization of the norms of game theory. This the- 
ory is described in chapter 9.2 by Andrés Perea. A differ- 
ent perspective is offered by evolutionary game theory, 
which is outlined by J. McKenzie Alexander in chapter 9.3. 
Again, these theories are criticized from the empirical 
point of view. This led to the rise of behavioral and psy- 
choeconomics, which are sometimes (mistakenly) seen as 
fields of psychology but are still driven by the interests of 
economists. The many different ideas in this area are pre- 
sented in chapter 9.4 by Sanjit Dhami and Ali al-Nowaihi. 

In our introductory chapter, we explain why most 
research on rationality focuses on individual rationality. 
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In fact, section 10 is the only section that really deals 
with social rationality in the proper sense. Therefore, 
it is not strictly limited to practical matters. In chap- 
ter 10.1, Franz Dietrich and Kai Spiekermann deal with 
social epistemology, which discusses the normative stan- 
dards for group belief formation. Recently, related topics 
such as we-intentionality and collective rationality have 
developed into a larger philosophical field. These topics 
are presented in chapter 10.2 by Hans Bernhard Schmid. 
Communication, particularly linguistic communication, 
is a very special social and rational activity, which is the 
topic of chapter 10.3 by Georg Meggle. Rational choice 
theory has also become a strong paradigm in the social 
sciences. This line of research is explained by Werner 
Raub in chapter 10.4. A look into the history of philoso- 
phy, into political philosophy, and so on reveals that the 
standard theories of practical rationality go far beyond 
the scope of instrumental decision making. At least 
chapter 10.5 by Julian Nida-Riimelin, Rebecca Gutwald, 
and Niina Zuber discusses such extensions and under the 
label of “structural rationality.” Finally, in chapter 10.6, 
Leda Cosmides and John Tooby present their adaptation- 
ist account of rationality. Although their evolutionary 
account seeks to explain all forms of human rationality, 
their chapter mainly focuses on practical rationality. 
Practical reasoning need not proceed in quantita- 
tive terms of probabilities, utilities, or the like. Deontic 
and legal reasoning, for instance, can rely on qualitative 
representations and processes. Such qualitative accounts 
are the topic of section 11. In chapter 11.1, John Horty 
and Olivier Roy present the current state of the art in 
deontic logic (i.e., the logic of obligations and permis- 
sions). Then, in chapter 11.2, Shira Elgayam deals with 
the psychological theories of deontic reasoning. Legal 
reasoning is sufficiently different from deontic logic 
to deserve a separate treatment. This is given in chap- 
ter 11.3 by Eric Hilgendorf and in chapter 11.4 by Henry 
Prakken. A surplus of these chapters is that they represent 
different legal cultures, which heavily determine the 
field: chapter 11.3 is embedded in German law, while 
chapter 11.4 is based on the Anglo-Saxon legal system. 
Whenever we are engaged in practical reasoning and 
decision making, moral issues come into play. In fact, 
they often are more important than issues of rational- 
ity. However, morality is a different topic, which we do 
not tackle in this handbook. We only wanted to address 
a few connections to rationality. These connections are 
the topic of section 12, which consists of two philosoph- 
ical chapters and a psychological one. Chapter 12.1 by 
Christoph Fehige and Ulla Wessels describes the relation 
between rationality and morality in philosophy, while 
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chapter 12.2 by Michael Smith deals with moral reasons, 
also from a philosophical point of view. But moral judg- 
ments are also a big topic in psychology. In chapter 12.3, 
Alexander Wiegmann and Hanno Sauer describe the psy- 
chological view on moral judgments and their relation 
to rationality. This concludes our chapters on practical 
rationality. 


Part IV: Facets of Rationality 


Sections 13-15 deal with further facets of rationality. 
Their topics are so distinct that they could not be sub- 
sumed under the headings of theoretical and practical 
rationality. In fact, most of the questions arise in both 
areas of human reasoning. For example, section 13 is 
concerned with visual and spatial thinking. Such cog- 
nitive processes are an important complement to the 
many chapters in which reasoning is conceptualized, 
more or less explicitly, in linguistic or propositional terms. 
An important question in this context is the connection 
between logical and diagrammatic reasoning. In chap- 
ter 13.1, Mateja Jamnik takes the perspective of a com- 
puter scientist and argues that diagrams can help to 
reason logically. Another question is how humans and 
artificial intelligence systems reason about space and 
time. In chapter 13.2, Marco Ragni describes some dif- 
ferences and explains why people sometimes commit 
errors in spatial and temporal reasoning. In chapter 13.3, 
Markus Knauff explores whether visualization supports 
or hampers human reasoning. The empirical results 
show that it can indeed sometimes impede reasoning. 

Another distinct topic is scientific rationality, which 
has theoretical as well as practical aspects, and which is, 
of course, of essential concern to us as scientists. The 
three chapters in section 14 deal with different aspects 
of scientific rationality. One idea is that the sciences 
exercise a particularly sophisticated form of epistemic 
rationality, which may have stronger claims on objectiv- 
ity. This idea is pursued in chapter 14.1 by Line Edslev 
Andersen and Hanne Andersen. Another topic is the value- 
freedom of science. This is the topic of chapter 14.2 by 
Anke Bueter. Finally, it is also important how scientific 
results are communicated to and perceived by the pub- 
lic. In chapter 14.3, Rainer Bromme and Lukas Gierth dis- 
cuss the public understanding of science and its relation 
to scientific rationality. 

The final section 15 consists of still more chapters of 
psychological and even political interest. Chapter 15.1 
by Henry Markovits explains how children learn to rea- 
son rationally. Chapter 15.2 by Keith Stanovich, Mag- 
gie Toplak, and Richard West presents the psychological 
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findings concerning the connection between intelli- 
gence and rationality—it is not quite as close as it may 
seem. Chapter 15.3 by Stephanie de Oliveira and Rich- 
ard Nisbett, finally, reports how training can improve 
people’s rational thinking and that such training does 
not even need to be intensive or time-consuming. 

This is also an optimistic message at the end of this 
handbook, which offers a big journey through the state of 
the art in rationality research. We can only recommend 
again to start this journey with our introductory chapter. 
It gives a systematic structure to the field and should 
help our readers to make cross-connections between the 
philosophical and psychological methods, theories, and 
ideas represented in this handbook. Research in these 
(and some other) areas has been conducted separately 
for too long, an anachronism that this handbook aims 
to overcome. 
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Summary 


Rationality is a vast issue. It has occupied the most bril- 
liant thinkers since the beginning of human culture. 
It mirrors our capacity for self-reflection, our desire to 
recognize the potentials and limitations of our mental 
abilities, and our determination to understand how our 
mind works. Such questions can be frightening, as they 
seem so complex that we may never find conclusive 
answers. But we can try. In this introductory chapter 
to The Handbook of Rationality, we unfold the concept 
of rationality and the relations between theoretical and 
practical rationality, normative and descriptive theories 
of rationality, individual and social rationality, and an 
output-oriented and a process-oriented perspective on 
rationality. We also describe some cognitive precondi- 
tions and domains of rationality and the different intel- 
lectual traditions in psychology and philosophy—where 
they intersect, fall apart, and converge. We hope that 
this overview helps readers to orient themselves in the 
widely ramified research on human rationality. 


1. Why Study Rationality? 


Humans are rational animals. Since Aristotle, this has 
been considered the essence of Homo sapiens. Of course, 
much about this view can be criticized: one objection 
might be that irrationality seems to prevail. The human 
species is about to make our planet uninhabitable. The 
globe is full of political, economic, cultural, and reli- 
gious conflicts. Often, people seem to be driven not by 
rationality but rather by power struggles, anxiety, greed, 
prejudice, and so on. All of this can scarcely be called 
rational. However, it is hard to argue about the rela- 
tive amounts of rationality and irrationality. Rational- 
ity attributions are delicate, as this handbook will richly 
display. We well know that a collective of rational indi- 
viduals may end up in an irrational collective disaster 
(Janis, 1972; Moorhead, Ference, & Neck, 1991; Reason, 
1987). And the extent of the irrationality often depends 


on the perspective of the assessor. In fact, to call humans 
rational is to say that they are gifted with rationality, but 
not that they exercise this gift all the time. In Chomsky’s 
terms, rationality is a competence, and the performance 
may well be defective. 

A second objection to the definition of humans as 
rational animals might be that rationality is not unique to 
humans. Today we know that many other animals have 
mental capacities that were traditionally assumed to dis- 
tinguish Homo sapiens. Of course, it is important to study 
continuities and differences between potentially rational 
creatures, and there is no doubt that human rationality 
has evolved from more fundamental neural mechanisms 
of adaptation and cognition. However, our mental pow- 
ers of representation, of differentiation, of inner and outer 
experience, are much larger than those of any other ani- 
mal. We find full-blown rationality only in us, although 
it also exists in basic forms in other species. 

A third objection to the definition of humans as ratio- 
nal animals might be that various other characteristics 
distinguish our species from other animals as well: lan- 
guage, religion, pedagogics, arts, aesthetics, humor—the 
list is endless. What is so special about rationality? Take 
language, for instance: one might say that language is not 
only for communication but also for so many other social 
and individual purposes. Hence, compared to rationality, 
it is the far more comprehensive phenomenon. However, 
the many uses of language depend on the serious and 
sincere exchange of information and expression of beliefs 
and desires (see D. Lewis, 1969, and chapter 10.3 by 
Meggle, this handbook). Without the formation of such 
beliefs and desires—an essentially rational process— 
language would not be what it is. 

Still, some of these objections may be justified. How- 
ever, if we really were to doubt that rationality is at the 
heart of our existence as human beings, we would have 
difficulties explaining how people think, reason, judge, 
make decisions, solve problems, and act. Only on this 
rationality assumption can we explain how we inter- 
act, communicate, and form societies. And only on this 


assumption can we explain how humans developed sci- 
ence, culture, and modern technology. All these abilities 
and achievements rely on our mental powers of repre- 
sentation and processing, which are implemented in the 
neural structure of our brains. Thus, it is legitimate to 
question how rational humans are, but shifting priority 
away from rationality would be a mistake. 

Today we are facing many vital challenges, from fight- 
ing starvation and curtailing war to coping with climate 
change, pandemics, worldwide diseases, and the draw- 
backs of new technologies and globalization. Indeed, for 
the first time in history, it seems that these challenges 
may concern humankind globally and substantially. 
One may well argue about what will help us most in 
meeting these challenges: empathy, justice, modesty, 
peacefulness, honesty, solidarity, the right values, reli- 
gion? Surely rationality alone won’t do. Yet there is no 
way to bypass rationality. Theories of justice refer to the- 
ories of rationality; we must act rationally to reach our 
moral, ethical, and humanist goals; and even religion 
cannot ignore rationality. So, whichever human features 
we promote, whatever measures we propose in order to 
meet our challenges, they should be well reasoned and 
withstand our critical reflection. 

These first thoughts already reveal the two funda- 
mental aims of this handbook. One aim is to bring 
together mainly psychological and mainly philosophi- 
cal accounts of rationality and to display what both dis- 
ciplines can learn from each other. The other aim is to 
substantiate the relation between the normative and the 
descriptive perspective on human rationality,’ between 
what human thinking ought to be like and what it actu- 
ally is like. 

One might say that philosophy has been attempting 
to grasp rationality for 2,500 years, while psychology has 
been doing so only since the end of the 19th century. But 
this does not adequately represent the intellectual his- 
tory. Large parts of this history, beginning already with 
the Presocratics (see Lorenz, 2009), were concerned with 
psychological issues, even if called philosophy, and were 
indeed labeled “philosophical psychology.” Yet, for a 
long time, the methodology of this field was not clear, 
and hence the normative and the descriptive perspec- 
tive were often not clearly defined and separated (see 
chapter 1.1 by Sturm, this handbook). This changed at 
the end of the 19th century, when psychology became 
an independent academic discipline, and one of its first 
topics was indeed to empirically investigate the specific 
conditions under which cognition and action conform 
to or deviate from what were at that time regarded as the 
norms of rationality (Storring, 1908; Wundt, 1896/2010). 
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Interestingly, while this happened in scientific psychol- 
ogy, logic developed in the opposite direction and became 
determinedly antipsychologistic under the influence of 
Frege (1884). We will return to this below. 

Nowadays, the roles of psychology and philosophy in 
the academic landscape are clarified. Psychology is now 
the empirical science of the mind, with its own meth- 
odology, while philosophy sees more clearly that the 
normative issues belong to its proper domain. Perhaps 
because each discipline first had to find its role, the dis- 
course between the two disciplines was initially poor or 
appeared quite unproductive. However, this has recently 
changed to a high degree, as will be amply displayed in 
this handbook.” 

Clearly, both perspectives, the descriptive and the 
normative one, are each important and valuable in 
themselves. Their relation, however, is utterly contested. 
Some scholars argue that psychology should ignore the 
normative work in philosophy and simply go ahead 
describing how people think and reason (Elqayam & 
Evans, 2011). Other scholars refer to the fundamen- 
tal distinction between competence and performance 
and argue that it is in principle impossible to empiri- 
cally confirm that humans are irrational (Cohen, 1981). 
If either of these camps is right, there is no tension 
between the normative and the descriptive perspective 
on rationality—they are either independent or basically 
coincide. However, both views are minority positions 
in the community of rationality researchers. The prem- 
ise of this handbook is instead that a comprehensive 
account of human rationality requires both empirically 
evaluated descriptive theories and elaborated normative 
theories as a positive or a negative point of comparison. 

The aim of this handbook is to give a comprehensive 
overview of the state of the art in philosophy and psy- 
chology, and of the areas where the two disciplines are 
connected, where they share interests and concepts, and 
where they have diverging interests and research agen- 
das. The handbook has many contributions from both 
disciplines and also includes some approaches from eco- 
nomics, sociology, artificial intelligence, and cognitive 
neuroscience, even if it cannot do so exhaustively. 

This introductory chapter provides some structure to 
our huge topic. It does not follow the table of contents, 
which we already explained in the overview of the hand- 
book. Rather, the arrangement of the chapter reflects some 
basic distinctions, conceptual frameworks, and research 
lines in the philosophical and psychological exploration 
of human rationality. It is organized as follows: 

In section 2 of this chapter, we start with a prelimi- 
nary contour of the subject matter we are talking about 
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when we use the term “rationality.” Then, in section 3, 
we describe the essential distinction between theoretical 
or epistemic and practical rationality. Psychologists are 
used to making a related distinction between reasoning 
and decision making. Next, in section 4, we will discuss 
the important tension between a normative and a descrip- 
tive perspective on rationality. As we just stated, this 
is one of the most crucial issues in our psychological- 
philosophical enterprise. In section 5, we introduce the 
distinction between individual and collective or social 
rationality, a topic that overlaps with topics from eco- 
nomics and other social sciences. Section 6 is concerned 
with the distinction between what we call output-oriented 
and process-oriented theories of rationality. While the for- 
mer is the dominant approach in philosophy, the latter 
lies at the heart of cognitive psychology, which seeks to 
understand the processes that lead from the input given 
to the cognitive system to the output generated by the 
system. The four distinctions in sections 3 to 6 can be 
combined with each other and thus lead to a systematic 
framework that, for the first time, combines philosophi- 
cal and psychological theories and empirical results on 
human rationality. We think that this framework not 
only guides the readers through this handbook but also 
provides a new perspective on research on human ratio- 
nality. Then, in section 7, we will be concerned with 
some of the preconditions of human rationality, that is, 
the neural underpinnings of thinking and reasoning, as 
well as the relation between rationality and intelligence, 
memory, and other cognitive functions. In section 8, 
we discuss whether special forms of rationality exist, for 
example, in science, communication, or artificial intelli- 
gence (AI). We also discuss some connections of rational- 
ity to concepts such as emotion, morality, and culture. 
In the (final) section 9 of this chapter, we conclude with 
some open questions and problems that interdisciplin- 
ary rationality research still has to tackle. 


2. Contours of the Concept of Rationality 


Rationality constitutes a rich conceptual field. Broadly 
construed, it is about our higher cognitive faculties, about 
acquiring concepts, forming beliefs, gaining knowledge, 
inferring and reasoning, thinking, judging, decision 
making, planning, deliberating, calculating, and satisfy- 
ing wishes, needs, and desires. In recent terms, we may 
also say that it is about our intentional and propositional 
attitudes. Being endowed with self-reflection, humans 
have surely wondered about these issues all along and 
so started thinking about how the human mind might 
work. 


The field is also most confusing, tentatively tamed 
by many incongruous terms. If we look only at the five 
languages in which essential parts of early philosophi- 
cal psychology were conducted—Greek, Latin, French, 
English, and German—we find many relevant terms, 
often accompanied by full doctrines. These terms are 
neither easily translatable, nor do they have a stable 
meaning. Think, for instance, of the distinction between 
understanding and reason, which acquired ever more sig- 
nificance in 17th- and 18th-century philosophy, culmi- 
nating in Immanuel Kant, for whom that distinction 
between “Verstand” and “Vernunft” took a very specific 
form closely intertwined with his entire philosophical 
edifice. In the hermeneutic tradition, then, understand- 
ing was rather something opposed to explanation (von 
Wright, 1971). Nowadays, these distinctions are treated 
as spurious by most authors, with the exception, per- 
haps, of orthodox Kantians. 

Or consider the English term “reason,” which has three 
different meanings, for which German, for example, uses 
three different words. First, “reason” without a determiner 
(Vernunft) is just a general colloquial term for our higher 
cognitive faculties (which derives from medieval German 
vernehmen which meant erfassen, i.e., “to grasp”). Second, 
“a reason” (Grund) is what we have or give or accept in 
order to explain or justify whatever may be explained or 
justified. And finally, “to reason” (rdsonieren, schliefsen) 
is something we do when we argue, make inferences, 
or arrive at conclusions. The connection presumably is 
that when we reason or give reasons, we use our reason. 
However, when we give reasons, we are not necessarily 
reasoning, and reasoning need not proceed from reasons 
(but only from premises the status of which may be left 
open). That’s the English muddle, which will occupy us 
later. Other languages have their own difficulties. 

The etymological origin of these terms is the Greek 
raetos, which, among other things, means “rational,” in 
the sense of rational numbers, and thus displays another 
aspect of our conceptual field. The term ratio owed its 
tremendous career also to the fact that Cicero (106-43 
BC) established it as the standard translation of the Greek 
logos. The term rationalitas, however, is first found in Ter- 
tullian (ca. 150-220 AD), for whom rationality was one of 
the essential attributes of the soul. Since possessing ratio- 
nality simply amounts to being endowed with reason, 
the term ratio/“reason” was the widely used one, while 
rationalitas/“rationality” played a minor role in history.* 

The term “rationality” became fashionable only at the 
end of the 19th century under the influence of econom- 
ics and social science. In philosophy, this was perhaps 
due to the fact that the normative dimension was seen 


more clearly in the 20th century and was more closely 
associated with the new term “rationality” than with old 
terms like “reason” and “understanding.” In psychology, 
the term became important in controversies about the 
role of empirical psychological laws for logic and epis- 
temology (Wundt, 1910). More details on the histori- 
cal developments in philosophy and psychology are 
represented in chapter 1.1 by Sturm and chapter 1.2 by 
Evans (both in this handbook).* In the following, we just 
describe some milestones in the recent history of our 
topic that were particularly influential in shaping the 
current state of the art and still guide ongoing debates in 
philosophy and psychology. 


2.1 Milestones in the Study of Rationality 

Let us start our milestones with George Boole and Gottlob 
Frege, two eminent philosophers and logicians. Boole 
(1854/1951) developed a new logic, now called “propo- 
sitional calculus,” which forms the basic part of classical 
logic. The title of his book, An Investigation of the Laws 
of Thought, indicates that Boole was concerned with 
human thinking, not with the abstract truth of proposi- 
tions. He wrote, “The design of the following treatise is 
to investigate the fundamental laws of those operations 
of the mind by which reasoning is performed” (Boole, 
1854/1951, p. 1). 

Frege later argued in exactly the opposite direction and 
became one of the most influential thinkers in the history 
of logic and philosophy. One reason for the importance 
of Frege’s work is that his revolution of logic was not 
only a revolution of large areas of philosophy. It was also 
the seed of what later emerged as analytic philosophy. 
Another reason is that his famous antipsychologism was 
essential for the separation of philosophy and psychol- 
ogy, a gap that we want to bridge in this handbook. In his 
Grundlagen der Arithmetik, Frege suggested, as one of three 
fundamental principles,* that one must “separate sharply 
the psychological from the logical, the subjective from 
the objective” (Frege, 1884, p. x). For Frege, grasping a 
thought such as Newton’s law of gravitation “is a process 
which takes place at the very confines of the mental and 
which for that reason cannot be completely understood 
from a purely psychological standpoint. For in grasping 
the law something comes into view whose nature is no 
longer mental in the proper sense, namely the thought” 
(Frege, 1979, p. 145). Here, one must observe Frege’s 
peculiar use of “thought.” For him, thoughts are objec- 
tive entities belonging to the abstract, nonmental “third 
realm” of “senses” (i.e., meanings). Thus, logic is about 
the laws of truth, which resemble laws of nature. These 
laws entail laws of thought. “Rules for asserting, think- 
ing, judging, inferring follow from the laws of truth” 
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(Frege, 1918, p. 289). Here, “rules” translates Vorschriften; 
they are prescriptions. So, Frege’s main claim was that 
the laws of thought are normative laws, not empirical 
laws of nature. 

In the same year in which Frege published his Be- 
sriffsschrift, namely in 1879, Wilhelm Wundt founded 
the first German institute for experimental psychology 
at the University of Leipzig. Wundt, who called Frege’s 
antipsychological approach logicism, argued that logic 
is a result of psychological functions and thus a branch 
of psychology. Although already the philosopher David 
Hume took psychology to be the backbone of philoso- 
phy, Wundt was probably the first psychologist who sys- 
tematically applied this view to the study of logic and 
rationality (Wundt, 1910). 

Wundt still retained the connection of psychology to 
philosophy. However, his work was also a milestone with 
regard to the academic independence of psychology. One 
of his many scholars, Gustav W. Stérring, may be said to 
have started empirical research on cognition and rational- 
ity when he published a 127-page article (Stérring, 1908) 
in which he described the first systematic experiments 
on human reasoning. In his studies, volunteers had to 
solve a battery of deductive reasoning problems, while 
their verbal responses, response times, eye movements, 
gestures, and even the expansion and contraction of their 
chest were measured. This work was part of Stdrring’s 
attempt to develop psychology into a more experimental- 
physiological direction. He was occupied with the laws of 
thought in a descriptive, not a normative, sense. 

Probably one of the most influential thinkers about 
the nature of human rationality was the developmental 
psychologist Jean Piaget. In his quite extensive publica- 
tions, Piaget placed great importance on the develop- 
ment of rationality in children, culminating in his 
pioneering book The Growth of Logical Thinking (Inhelder 
& Piaget, 1955/1958). His approach was often referred 
to as “genetic epistemology,” as he explained how 
people acquire knowledge and reasoning abilities by the 
biological functions of accommodation and assimilation 
to the environment. According to Piaget, people develop 
cognitively from birth throughout their lives in four dif- 
ferent stages: the sensorimotor (birth to age 2), preop- 
erational (2-7 years), concrete operational (7-11 years), 
and formal operational (11 years and onward) stages. 
For Piaget, rational reasoning is practically identical with 
logical reasoning and develops primarily in the formal 
operational phase, where children learn to think about 
abstract entities and concepts and develop logical abili- 
ties for reasoning, planning, and problem solving. For 
Piaget, these rational abilities rely on abstract formal 
rules of inference stored in long-term memory, which he 
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thought to be almost equivalent to the rules of formal 
logic. Later, this approach was developed further in rule- 
based theories of reasoning, one of which is described in 
chapter 3.2 by O’Brien (this handbook). This approach, 
however, has been attacked by the adherents of several 
other theories, notably the theory of mental models, which 
has been developed by Philip Johnson-Laird from the 
1980s onward and is another milestone in the history 
of the psychology of reasoning (Johnson-Laird, 1983, 
2006, 2010; Johnson-Laird, Khemlani, & Goodwin, 
2015). This theory is described in detail in chapter 2.3 
by Johnson-Laird (this handbook; see also chapter 3.3 by 
Khemlani and chapter 6.3 by Byrne & Espino, both in 
this handbook). 

Another major episode in the discussion of rationality 
was the Popper—Kuhn controversy. With Frege’s new logic as 
a firm base, Karl Popper (1934/1989) developed his falsifi- 
cationist hypothetico-deductive scientific methodology, 
which purported to state how science has to rationally 
proceed. Popper had a dispute about this with his positiv- 
istic colleagues in the Vienna Circle (e.g., Rudolf Carnap). 
And there was further controversy, notably in the social 
sciences, where the Frankfurt school (Habermas, Adorno, 
etc.) accused Popper of being a positivist. However, the 
dispute widened dramatically when Thomas S. Kuhn 
(1962) meticulously showed that scientists actually do 
not follow Popper’s methodology. One may conclude, 
as Popper did, that scientists proceed irrationally. But 
this is implausible. Isn’t rationality the hallmark of the 
sciences? Thus, Kuhn’s observations were taken rather 
as an empirical argument against Popper’s normative 
methodology—a most remarkable fact—and as a chal- 
lenge to find a more adequate normative methodology 
that does not accuse scientists of being irrational. Imre 
Lakatos’s (1978) “methodology of scientific research pro- 
grammes” was such an attempt. This gave rise to what 
is nowadays called “science and technology studies,” an 
amalgam of history, sociology, and philosophy of sci- 
ence. However, the controversy also motivated more 
abstract investigations into the logic of theory change 
and the nature of defeasible reasoning, which surface in 
many places in this handbook. 

Popper’s logic-inspired falsificationism was a direct ref- 
erence point for another milestone of cognitive rational- 
ity research: Wason’s selection task, which is sometimes 
called “the Drosophila of the psychology of reasoning” 
(Beller, 1999). The task is named after Peter Wason, a 
cognitive psychologist who wanted to explore whether 
people indeed seek to falsify logical rules as required by 
Popper (Wason, 1966, 1968). The task is as follows: 


You are shown a set of four cards placed on a table, each of 
which has a number on one side and a letter on the other 


side. The visible faces of the cards show A, D, 4, and 7. The 
cards obey the rule: if there is a vowel on the one side of the 
card, there is an even number on the other side of that card. 
Which card(s) must you turn over in order to test the truth 


of this rule? 
A D 4 7 


According to the norms of propositional logic, par- 
ticipants should check the validity of the given rule 
by turning over the cards with the A (modus ponens) 
and the 7 (modus tollens) on the front. Typically, fewer 
than 10 out of 100 participants can solve this problem 
correctly (J. St. B. T. Evans, Newstead, & Byrne, 1993). 
Why is that so? Psychologists have given many different 
answers to this question, some of which are discussed 
in chapter 1.3 by Schurz, chapter 2.3 by Johnson-Laird, 
chapter 4.5 by Chater and Oaksford, and chapter 15.1 by 
Markovits (all in this handbook). 

Such tasks were enormously important for the psy- 
chology of reasoning. They were designed to study pure 
inference processes without the contamination of true 
or false prior beliefs. The participants were instructed 
to draw logically valid conclusions from the premises, 
which they had to consider as facts that could not be 
questioned. Since the content of the premises did not 
matter, the only possible error seemed to be that the 
conclusion was inferred from the given premises in an 
unreasonable way. Although this paradigm has many 
advantages, it also led researchers to pay little attention 
to the contents of the beliefs represented in the prem- 
ises. Participants are simply forced to believe that the 
premises are true; otherwise their responses are often 
excluded from the analysis. 

Later, the pendulum swung more toward the study 
of beliefs and away from processes of reasoning. This 
development proceeded roughly in two steps. The first 
step consisted in research paradigms invented to study 
interactions between logicality and believability. In such 
experiments, a conclusion that is logically valid or invalid 
in relation to the premises could be true or false in rela- 
tion to the participants’ prior beliefs. This introduces the 
belief bias, which arises if a participant in an experiment 
infers a logically invalid conclusion from the premises 
because it agrees with her prior beliefs or, conversely, if 
she rejects a conclusion because it is implausible, even 
though it is logically valid (J. St. B. T. Evans, 1989). Such 
content-effects became very prominent in cognitive psy- 
chology and have been identified in almost all areas of 
human reasoning (J. St. B. T. Evans, 1993). 

The second step in developing more belief-related 
reasoning research was even more fundamental. Under 


the label new paradigm psychology of reasoning (Oaksford 
& Chater, 2007, 2020), researchers started around the 
year 2010 to focus on the subjective degrees of belief 
that people might have in the contents of the premises 
and on how those affect the acceptability of the con- 
clusion. Here, a different connection between psychol- 
ogy and Kuhn’s theory of progress in science shows up. 
Proponents of this approach see their new paradigm as 
“revolutionary” in the Kuhnian sense,’ as it replaces the 
norms of two-valued classical logic by those of Bayesian 
probability theory (Oaksford & Chater, 2007). While the 
previous research regarded the influence of beliefs as a 
bias, the new paradigm puts such subjective degrees of 
belief center stage. The positions of the leading figures 
in the field are represented in chapter 4.5 by Chater and 
Oaksford and chapter 6.2 by Over and Cruz (both in this 
handbook). 

These developments in psychology go back to another 
milestone on the normative side of rationality: the devel- 
opment of probability theory by Reverend Thomas Bayes 
and Frank P. Ramsey. Thomas Bayes (1764/1970) is often 
credited with having established an important theorem 
in his groundbreaking work (but see Stigler, 1983), a the- 
orem showing how to infer new posterior probabilities 
from old prior probabilities and the evidence. Today, this 
framework is called Bayesianism and is very popular in 
philosophy and psychology. An important later step was 
the work by Ramsey, who—in his 1926 essay “Truth and 
Probability” (in Ramsey, 1978)—strongly emphasized the 
subjective interpretation of probability. Unfortunately, 
Ramsey’s writings are few, since he died in 1930 at the 
age of 26. However, the widespread popularity of Bayes- 
ianism in epistemology, philosophy of science, statistics, 
and cognitive psychology owes much to Ramsey, who 
made degrees of belief central to epistemology. Ramsey 
also insisted that degrees of belief must be conceived as 
subjective probabilities and even proposed, for the first 
time, a method for measuring those subjective degrees 
of belief. In all fairness, though, Bruno de Finetti (1937) 
must be mentioned as another pioneer in subjective 
probability theory. (See also Gillies, 2000; Krtiger, Das- 
ton, & Heidelberger, 1987; Kruger, Gigerenzer, & Mor- 
gan, 1987.) 

Nowadays, most psychological research in the “new 
paradigm” is within this Bayesian framework (see the 
chapters in section 4 of this handbook).’ Slowly, how- 
ever, the insight gains strength that subjective degrees of 
belief do not need to be modeled as probabilities. Models 
can also build on other systems for representing degrees 
of belief and their revision. These options will be richly 
dealt with in this handbook from both a normative and 


Markus Knauff and Wolfgang Spohn 


a descriptive point of view (see chapter 4.7 by Dubois & 
Prade and the chapters in section 5 of this handbook). 
Ramsey was also important for another paradigm 
of rationality research, namely decision theory, which 
builds on probability theory and assumes decisions and 
actions to be guided by utilities. In his paper “Truth and 
Probability,” he proposed for the first time a method 
for simultaneously measuring subjective probabilities 
and utilities, and thus laid the foundations of standard 
decision theory as it is taught today. This measurement 
method also offered novel ways of justifying decision 
theory, which later gained ever more importance (see 
chapter 8.2 by Peterson, this handbook). And this was 
only possible by interpreting probability in a subjectiv- 
ist way. Nowadays, standard decision theory is the most 
widely accepted account of practical rationality and 
decision making. It serves as the basis of economics and 
of much of the social sciences, at least as far as it is pur- 
sued within the rational choice paradigm (see sections 8 
and 9 and chapter 10.4 by Raub, all in this handbook). 
Another milestone in rationality research was the 
work of Herbert A. Simon (Simon, 1957, 1959), who was 
as strongly influenced by standard decision theory as 
Wason was inspired by Popper’s falsificationism. Simon 
criticized standard decision theory for its idealizations 
and claimed that human decision making is limited in 
various ways: by incomplete knowledge, by imperfect 
memory, by restricted capacities of representation and 
computation, and so on. Thus, he developed the con- 
cept of bounded rationality, which has by now ramified 
into various research fields. Simon was also the first to 
suggest that human behavior is determined by heuristics 
rather than by a decision calculus. He proposed the satis- 
ficing heuristic, according to which the search for options 
is stopped as soon as an option is found that reaches a 
preset achievement level such that no further optimiza- 
tion is needed. Simon’s work first radiated into econom- 
ics but was soon recognized in cognitive psychology, 
where it initiated much experimental work (Gigerenzer 
& Selten, 2002). As a pioneer of cognitive-oriented arti- 
ficial intelligence (together with Allen Newell, he inven- 
ted the general problem solver [GPS]), Simon was also a 
founding father of cognitive science (Newell & Simon, 
1972), where rationality research looms large as well. 
The idea of heuristics was extended in the work of 
Amos Tversky and Daniel Kahneman (1974), two psy- 
chologists who were also interested in economics. Their 
experiments showed that human judgment does not 
conform to the rules of the probability calculus but is 
rather guided by various heuristics, cognitive rules of 
thumb that are often helpful because they save cognitive 
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resources, but may also lead to systematic deviations 
from the norms of mathematical probability and deci- 
sion theory. This approach was further developed in the 
program of ecological rationality by Gerd Gigerenzer and 
coworkers, although in this theoretical framework, heu- 
ristics have a less negative connotation. In fact, these 
researchers argue that heuristics are highly adaptive and 
rational (Gigerenzer, Hertwig, & Pachur, 2011). Kahne- 
man and Tversky (1979) also developed prospect theory 
and presented many experimental findings showing that 
this psychological theory is empirically more adequate 
than standard decision theory from economics. Today, 
prospect theory and the heuristics and biases program can 
be found in most textbooks of psychology and behav- 
ioral economics (see chapter 8.3 by Glockner and chap- 
ter 8.5 by Hertwig & Kozyreva, both in this handbook). 

It should be noted, though, that most of the reported 
theories somehow oscillate between the normative and 
the descriptive view on rationality. Piaget was perhaps 
most explicit in using logic as both a descriptive and 
a normative framework for human rational reasoning. 
In contrast, standard decision theory was normatively 
motivated, and the hope was that it would not be too 
strongly empirically idealized. Kahneman and Tver- 
sky decidedly took the empirical point of view but also 
uncovered the many empirical inadequacies of norma- 
tive decision theory. Simon’s work was more ambiguous 
in this respect, as it attempted to define what rationality 
normatively requires, given the constraints of the cogni- 
tive system and the environment. Ecological rationality 
mirrors a similar view on human rationality. We shall 
return to these issues in section 4 of this introductory 
chapter and repeatedly in this handbook. Of course, 
these few milestones cannot stand in for a comprehen- 
sive history of our topic, not even for the past 150 years. 
But they should prepare the readers to better follow the 
further structure of this chapter. 


2.2 Basic Concepts of Rationality Assessment 

Let us turn to some basic concepts of rationality assess- 
ments. We already mentioned that the history of the ter- 
minology of rationality research does not provide a stable 
starting point. Neither is it advisable to start amid the cur- 
rent discussion—there is too much theory-ladenness. So, 
where should we start? Perhaps with ordinary language, 
which, albeit imperfect, is the best initial starting point 
we have: it preserves the insights of our ancestors, if also 
their confusions. Hence, let us briefly look at how we 
talk about rationality in our everyday life. Which are the 
things we call reasonable, rational, and so on? And what 
is it about those things that makes us call them rational? 


First, there are many things that are not subject to our 
rationality assessment, for instance, the weather. We may 
call them arational. They are outside of our consideration. 
Many things, though, are assessable as rational—and 
these kinds of things may also be assessed as irrational. 
For instance, people are upset about the irrationality of 
certain traffic regulations and road constructions, and the 
presentation of products in a supermarket may be very 
reasonable, at least from the manager’s point of view. Still, 
two categories stand out as primary objects of our ratio- 
nality assessments: beliefs or epistemic attitudes in general® 
and actions. This is why we carefully distinguish theo- 
retical rationality—which is concerned with beliefs—and 
practical rationality—which is concerned with actions 
(see section 3 of this introductory chapter). 

Other things may then be called rational or irratio- 
nal in a derived sense, insofar as they are caused by, or 
causally connected to, those primary objects, such as the 
road constructions or the presentation of products in the 
grocery store. Of course, we also assess persons and other 
animals as rational or irrational, insofar as they act and 
believe in a rational or irrational way. Whether we can 
evaluate emotions as rational or irrational as well is a 
matter of much controversy (see, e.g., de Sousa, 2011; 
D. Evans & Cruse, 2004; Helm, 2001). We will return to 
this question only briefly in section 8 of this introduc- 
tory chapter. Our overall focus is on what we just called 
the primary objects. We will explain below that inten- 
tions and desires should be included as well. 

We should emphasize, though, that it is actions and 
not behavior in general that are judged as rational. Psy- 
chologists typically say that action is behavior controlled 
by the mind. Philosophers typically say that action is 
intentional behavior (i.e., caused by an intention). Many 
psychologists study unintentional behavior, which is 
not, or hardly, under cognitive control. Sneezing, or sali- 
vation in the presence of food, is an arational, uninten- 
tional behavior. Sure, there is also intentional sneezing, 
which may or may not serve its purpose and may hence 
be assessed as rational—but this is fake sneezing. This 
restriction of rationality assessments to actions is impor- 
tant, but also delicate. Actions may be characterized as 
intentional and controlled even though the intention 
need not be consciously conceived and the control need 
not be actually exercised. So, actions may very well be 
purely habitual. Certainly, boundaries are vague. More- 
over, the intentionality of a behavior is only a necessary, 
not a sufficient, condition for its being an action.’ 

Let us look a bit more carefully at our rationality assess- 
ments. A noteworthy point is that our judgments can take 
a relative and a categorical form.'° In certain situations, 


your belief that it will rain this afternoon may seem obvi- 
ously unreasonable. This would be a categorical judg- 
ment. However, given that your only information is some 
outdated weather forecast, your belief may be reasonable 
after all, namely, relative to this information. Therefore, 
carrying an umbrella for your shopping tour may be cat- 
egorically called unreasonable, but relative to your unrea- 
sonable belief, it is perhaps not so unreasonable after all. 

Similarly, in psychological experiments, volunteers 
are typically instructed to assume that the premises of 
a reasoning problem are true. In the experimental set- 
ting, the premises are simply given and serve as input 
to be taken for granted by the participants. Thus, such 
experiments often assume a closed world, in which only 
the information from the premises is relevant for the 
conclusion. Relative to the closed world of the experi- 
ment, it would be reasonable to reach the conclusion. 
However, the assumption may be wrong. If the partici- 
pant reaches a different conclusion, this need not be 
unreasonable—it may be reasonable relative to the par- 
ticipant’s prior knowledge, which is not contained in 
the closed world but still taken to be relevant by the 
person. Psychologists then tend to speak of the belief 
bias (J. St. B. T. Evans, 1993), but in fact, such responses 
just demonstrate that human reasoning is often defea- 
sible and, thus, does not conform to the monotonicity 
principle of classical logic, according to which no valid 
inference can be turned invalid by simply adding prem- 
ises. Many experimental findings and different theories 
of defeasible reasoning are described in chapter 5.4 by 
Gazzo Castaneda and Knauff (this handbook). 

Relative judgments of rationality seem more basic, while 
categorical judgments seem derived: they are something 
like judgments all things considered or relative to the total 
evidence. We will yet look at how they might be derived. 
But let us first discuss what the relative judgments 
are relative to. In the case of beliefs, the answer seems 
straightforward: a belief is assessed as rational relative to 
the evidence the agent has, relative to her reasons or the 
information she has, or more generally relative to her 
other beliefs or the epistemic state the belief is embedded 
in. This raises the question whether a belief can also be 
rational relative to other than epistemic matters, desires 
perhaps, or emotions."! This is a difficult issue. It seems 
that a certain desire or emotion may be a cause of a belief 
but never a reason for it. We would disapprove of such a 
belief as wishful thinking. But what might be the psycho- 
logical mechanisms behind such phenomena? We shall 
return to this point in subsection 3.3 of this introductory 
chapter and when we discuss the distinction between 
the normative and the descriptive perspective and that 
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between output-oriented and process-oriented investiga- 
tions of rationality (section 6 of this chapter). 

If we accept that only beliefs can be reasons for a 
belief,'” it seems clear how we arrive at categorical judg- 
ments about beliefs: a belief is categorically reasonable if 
it is reasonable relative to other epistemic items that are 
in turn categorically reasonable. So, two things may go 
wrong with a belief: it may be based already on unrea- 
sonable premises, or it may be inferred from reasonable 
premises in an unreasonable way.’? Again, both can 
happen in psychological experiments: participants can 
either deviate from a logical conclusion because they do 
not accept the premises or proceed from other premises, 
or they take the premises for granted but commit errors 
in the reasoning process itself. 

So much for how we assess the rationality of beliefs. 
But what about actions, the other main class amenable 
to rationality assessments? Somehow our actions have 
to serve our goals according to our own lights. That is, 
the rationality of our actions is assessed relative to our 
beliefs and desires. This is the familiar and highly impor- 
tant notion of instrumental rationality, which is at home 
everywhere, in philosophy, psychology, and economics. 
“Beliefs and desires” must not be taken literally here. In 
fact, the term “beliefs” stands for an entire epistemic or 
cognitive complex, while “desires” stands for the aggre- 
gate of volitional, optative, conative, or buletic attitudes 
(“pro-attitude” is a more neutral but less familiar term). 
In ordinary language, we speak of drives, wishes, wants, 
interests, inclinations, goals or aims, norms and values, 
and the like. Psychologists distinguish between rela- 
tively stable behavioral dispositions, such as the needs 
for achievement, affiliation, or power, and short-term 
driving forces in a particular situation (Heckhausen & 
Heckhausen, 2018). However, let us here stick to “desire” 
as a generic term for this plenitude of concepts and clas- 
sifications for the driving forces for actions. 

Is there also a categorical rationality assessment of 
actions? Clearly, an action is categorically irrational if it 
derives from the given beliefs and desires in an irrational 
way. However, it may be categorically irrational even if 
it derives rationally from the given beliefs and desires. 
This would be the case if the beliefs or desires are them- 
selves irrational. We have already seen how beliefs may 
be judged to be categorically irrational. So, this judg- 
ment immediately enters the practical assessment of 
actions. But are desires themselves also subject to a ratio- 
nality assessment? Instrumental rationality refuses to 
judge desires in this way. It simply takes them as given. 
Recall the famous dictum of Hume (1739-1740/1975c, 
p. 415) that reason is and ought only to be the slave of 
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the passions. This leaves no room for a rationality assess- 
ment of the passions themselves. If so, the rationality of 
actions would ground only on the rationality of beliefs. 

However, the case is a bit more complicated. We need 
to distinguish between intrinsic desires, the objects of 
which are desired in themselves, and extrinsic desires, 
the fulfillment of which only serves some other (extrin- 
sic or intrinsic) desires. A similar distinction in moral 
philosophy is that between entities good as such, or as 
an end, and entities good as means. Relatedly, psycholo- 
gists distinguish between extrinsic and intrinsic motiva- 
tions. An action is extrinsically motivated when it serves 
to avoid something unpleasant or to get a return, a good 
grade, money, or the like. By contrast, it is intrinsically 
motivated when it is rewarding or enjoyable for its own 
sake, such as solving a puzzle or editing a book. 

With this distinction at hand, we can extend rational- 
ity assessments also to desires: extrinsic motivations or 
desires may be irrational given certain intrinsic motiva- 
tions or desires. If I want to become a millionaire (in 
order to satisfy my intrinsic desire for a carefree life) and 
therefore want to become a philosopher, then something 
is wrong with me. Either I have weird beliefs about the 
profitability of philosophy, or my leaning toward phi- 
losophy is simply irrational. That much we can say even 
from the point of view of instrumental rationality. 

This points to the real issue. Can we assess as rational 
also intrinsic desires, motivations, or values? Many ana- 
lytic philosophers still deny this and accept Hume’s posi- 
tion. They would only grant that actions and (intrinsic) 
desires can be judged as moral or immoral. However, the 
majority disagrees. A crucial point is that it becomes dif- 
ficult here to separate considerations of rationality and 
of morality (see chapter 12.1 by Fehige & Wessels, this 
handbook). The difficulty already commences with Kant, 
for whom the categorical imperative is the first a priori 
principle of practical reason. As such, it is a principle of 
rationality. But at the same time, it is the highest moral 
principle (see also chapter 1.1 by Sturm, this handbook). 
Max Weber (1921-1922) famously developed a concept 
of value rationality as opposed to instrumental rational- 
ity. These threads of Kant and Weber were taken up by 
many, notably by Jiirgen Habermas (1981), who sees 
practical rationality as going beyond instrumental ratio- 
nality, for instance, in communication, which is charac- 
terized by treating one’s fellows as ends in themselves (in 
chapter 10.3 in this handbook, Meggle argues that com- 
municative rationality is just instrumental rationality). 
Habermas (1973) criticized Weber’s concept of instru- 
mental rationality as a subordination to a fundamentally 
opportunistic lifestyle, while proper practical rationality 


also means becoming aware of and changing one’s own 
role in society. Quite different attempts to make sense of 
rational (and not already moral) assessments of intrinsic 
desires may be found in Nozick (1993), H. S. Richardson 
(1994), and Kusser and Spohn (1992). However, this is 
an obscure topic in philosophy and not treated in cogni- 
tive psychology. Therefore, we will not further pursue it 
in this handbook. In any case, it is important to be clear 
about the ways in which our rationality assessments are 
relative and can be categorical."* 

Having thus elucidated the objects of rationality 
judgments and their relational character, we have still 
not addressed the most essential question: What pre- 
cisely makes one object—belief, desire, action—rational 
relative to other objects? What is the content of this rela- 
tion? In short, what is rationality? In a way, the entire 
handbook is about this relation. Ordinary language is 
of little help here. We might say that the things relative 
to which some belief or desire or action is rational are 
the reasons for that belief or desire or action. But what is 
this reason relation? We certainly have a good intuitive 
understanding of it. We might say that some kind of rea- 
soning or inference leads from the former to the latter. But 
what are the mechanisms governing such inferences? 
Questions like these are omnipresent in this handbook. 

David Marr (1982) has famously introduced the dis- 
tinction between the computational, the algorithmic, 
and the implementational level of explaining cognitive 
phenomena, a distinction governing cognitive science 
ever since. On the computational level, the goal of a com- 
putation is set: what is its function, and what is to be 
achieved by it? The algorithmic level inquires how this 
computational goal is reached: which processes or algo- 
rithms are used to reach the goal defined on the compu- 
tational level? The implementational level then addresses 
how these processes are physically realized. Searle (1992) 
argued that mental processes can be explained only 
on the basis of the biological, not just the functional, 
properties of the brain, a position he calls “biological 
naturalism.” By contrast, functionalist philosophers of 
mind from Putnam (1967/1975) and Fodor (1975, 2000) 
onward have claimed that the implementational level 
is irrelevant for understanding the functioning of the 
mind: it is largely irrelevant whether the processes on 
the algorithmic level are realized in a biological system 
such as the human brain or by the silicon chips of a com- 
puter. And there are even modern dualists, like Chalmers 
(1996) and others, who argue that some mental char- 
acteristics cannot be reduced even to functional char- 
acteristics. We briefly discuss this topic in section 7 of 
this introductory chapter about the preconditions of 
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rationality. A more detailed description of the cortical 
basis of human rationality is given in chapter 1.4 by 
Goel (this handbook). However, in the main, this hand- 
book sticks to the computational and the algorithmic 
level of description, because we think that these are the 
most relevant levels if one really wants to understand 
what makes people rational—and sometimes irrational. 

However, Marr’s terminology is a bit confusing. His 
conceptual distinctions are immensely important, but 
the terms are misleading. For instance, a computation 
consists just in running an algorithm, and the algorith- 
mic level also determines the kind of representations on 
which the algorithm works. Therefore, we here use the 
terminology already indicated at the end of section 1 of 
this introductory chapter, where we called Marr’s com- 
putational level the output-oriented and the algorithmic 
level the process-oriented level of explanation.’* This ter- 
minology is less biased toward computer science and 
also signals a slight shift of meaning. In particular, we 
want to make room for the idea that the computational 
goals are set by normative and not by empirical consid- 
erations. We will return to this issue in section 6 of this 
introductory chapter. But before that, we now come to 
the top-level distinction in most conceptions of ratio- 
nality, in both psychology and philosophy. 


3. Theoretical and Practical Rationality 


Let us start with an example. Many current public and 
scientific debates revolve around the increased mortal- 
ity of honeybees. This Sunday, there is a demonstration 
against the use of the herbicide glyphosate, which is 
feared to be harmful to animals, including bees. Should 
you participate? If you want to make a rational decision, 
your thinking should proceed in two steps: in the first 
step, you should weigh the arguments that speak for or 
against the assumption that glyphosate kills bees, evalu- 
ate the inferences made by the supporters and opponents 
of this claim, consider empirical evidence from scientific 
research, and so on. Let us assume that, based on these 
considerations, you conclude that glyphosate indeed can 
cause increased mortality among honeybees. Now, in a 
second step, you have to decide whether or not you will 
go to the demonstration. On the one hand, you would 
prefer a relaxed Sunday on your sofa, it is likely to rain, 
and you fear that there may be some violence at the dem- 
onstration. On the other hand, you think, based on your 
previous conclusion, that glyphosate should be banned 
and that it is important to fight for nature, and many of 
your friends will be there too. Both considerations are 
important for your decision to attend the demonstration. 
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Of course, in daily life, these kinds of thoughts are not 
so clearly separated, your decision might be influenced 
by many other internal and external factors, and so on. 
Yet, our example makes clear that we must carefully dis- 
tinguish two aspects of rationality. The first part of the 
example refers to theoretical rationality, which is about the 
rational justification of beliefs, inferences, and explana- 
tions, or of our epistemic states in general. This issue lies 
at the center of epistemology and constitutes one of the 
main topics in the philosophy and psychology of rational- 
ity. Thus, the Wason selection task and the large amount 
of studies on human conditional, syllogistic, probabi- 
listic, counterfactual, causal, or relational reasoning are 
concerned with theoretical or epistemic rationality. The 
so-called Linda problem, which is almost as famous as 
the Wason selection task (see chapter 1.2 by Evans, chap- 
ter 4.3 by Merin, and chapter 8.5 by Hertwig & Kozyreva, 
in this handbook), is also concerned with the theoretical 
rationality, or irrationality, of human reasoners. 

The second part of the example above is concerned 
with practical rationality, which is about assessing actions 
or pro-attitudes in general. Here, psychologists explore 
how people choose between different alternatives that 
have different values for them. Thus, it is concerned with 
what people decide in a particular situation where differ- 
ent options are considered. 

John Searle, who is just as important for philosophy 
as for the cognitive sciences, expressed the distinction in 
terms of “direction of fit”: theoretical rationality is about 
how to represent the world, how to make our mind corre- 
spond to it, whereas practical rationality is about how to 
shape the world, how to make it correspond to our mind 
(Searle, 1983). The distinction can also be made with 
respect to the kinds of reasons that can be adduced for 
the attitudes in question. Beliefs, or epistemic attitudes 
in general, can only be justified with reference to fur- 
ther epistemic elements—knowledge, beliefs, evidence, 
perceptions, or testimony—while the reasons for actions 
and intentions lie not only in those epistemic elements 
but also in our motives and desires or pro-attitudes. So, 
to account for theoretical rationality, we need to talk 
only about epistemic matters, whereas accounting for 
practical rationality also requires talking about motiva- 
tions, aims, values, and so on. 

Some psychologists might have problems with this 
sharp separation of theoretical and practical rational- 
ity. Their main argument is that in daily life, the func- 
tions of reasoning and decision making are intertwined. 
Both kinds of rationality, theoretical and practical, are 
strongly interlinked, because reasoning typically serves 
good decision making. This is also substantiated in the 
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account of utility conditionals in chapter 6.4 by Bonnefon 
(this handbook). Recently, the proponents of the “new 
paradigm” in psychology have argued in this direction, 
as they consider subjective psychological value, or util- 
ity, and social pragmatics as important for reasoning. 
Accordingly, this approach aims to integrate the psy- 
chology of reasoning with the study of decision making 
(Elgayam & Over, 2013). 

It is certainly true that we can find reasoning on both 
sides. There is even an influential philosophical position 
to the reverse effect, namely, that we can find decision 
making on both sides: we decide not only what to do but 
also what to believe. This doctrine, doxastic voluntarism, 
goes back to Descartes (1641, Fourth Meditation). It tries 
to understand belief formation as deciding which beliefs 
best fulfill our epistemic aims, where truth is the central 
epistemic aim, but may be accompanied by other aims. 
The result is what is called epistemic decision theory (Konek 
& Levinstein, 2019; Levi, 1967). We can even say that par- 
ticipants in reasoning experiments decide between con- 
clusions having different values—a reasoner may value 
the valid conclusion more than the invalid one. In these 
psychological and philosophical lines of thought, there 
is no clear distinction between theoretical and practical 
rationality. 

Here we disagree. In our view, practical rationality pre- 
supposes theoretical rationality. We think that we can 
study theoretical rationality independently from practi- 
cal rationality but not the other way around. The reason 
is that to account for theoretical rationality, we need 
to talk only about epistemic matters, whereas account- 
ing for practical rationality also requires talking about 
motivations, aims, values, and so on. The activity of rea- 
soning is necessary on both sides. However, theoretical 
reasoning proceeds from factual or empirical premises 
to factual or empirical conclusions. Practical reasoning 
takes motives or values or pro-attitudes as additional 
premises and arrives at practical conclusions (intentions 
or actions). Of course, these are different reasoning tasks 
requiring different cognitive processes. In the following 
two subsections, 3.1 and 3.2, we unfold the distinction 
in more detail. In subsection 3.3 of this introductory 
chapter, we will return to the issue of what may be spe- 
cial about theoretical rationality. 


3.1 Theoretical Rationality 

Theoretical rationality is often also called “epistemic ratio- 
nality,” even though, as mentioned in subsection 2.2 of 
this introductory chapter, “doxastic rationality” would be 
the more appropriate term. There, the fundamental issue 
concerns the rational form of epistemic attitudes, states, 
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or processes. Concerning this shape, the first question is 
which entities those attitudes are about, that is, what it 
is that we believe, take to be plausible, and so on. Usu- 
ally, these are taken to be linguistic entities. And usually, 
these entities are taken not to be sentences or utterances 
themselves—otherwise the Frenchman and the Spaniard 
would not be able to share beliefs—but rather the contents 
or meanings of sentences and utterances. Philosophers 
and cognitive psychologists call the latter “propositions,” 
while there is some disagreement about a suitable char- 
acterization of propositions as objects or contents of 
epistemic attitudes.'° This handbook is silent on those 
issues—deliberately, because treating them fairly would 
mean engaging deeply the most fundamental questions 
of the cognitive sciences and the philosophy of mind and 
language, which are not our topic. The consequence of 
our decision not to enter into issues of mental contents is 
that this handbook remains quite vague about the objects 
or contents of our beliefs and, when it comes to practical 
rationality, about the contents of our motives and desires. 

The next important point pertains to the many pos- 
sible forms of epistemic states. Basically, these forms 
may be conceived in a qualitative, a comparative, and a 
quantitative way. In many situations in everyday life 
(and in the psychological lab), people just believe or dis- 
believe something, for example, that glyphosate is harm- 
ful to bees. For many decades, psychologists have mainly 
explored inferences based on such qualitative beliefs. 
Usually, participants had just two response options: 
belief or disbelief, entailed by the premises or not, logi- 
cally valid or invalid, yes or no, and so forth. Today we 
have a relatively clear empirical understanding of how 
people deal with such inferences, although there is less 
agreement regarding the underlying cognitive processes 
(J. St. B. T. Evans et al., 1993; Johnson-Laird & Byrne, 
1991). Thus, an epistemic state (of a certain person at a 
certain time) may be qualitatively characterized simply 
as a Set of beliefs, a set of contents that are accepted (or 
“endorsed” or “maintained”) by the person at that time. 
If this set may be an arbitrary set, philosophers call it a 
belief base; if it is to satisfy the fundamental and strong 
but contested rationality postulates of consistency and 
deductive closure, it is called a belief set. Philosophers also 
discuss weaker rationality requirements. This conception 
is unfolded in several chapters of this handbook (see in 
particular chapter 5.1 by van Ditmarsch and chapter 5.2 
by Rott). 

However, this characterization neglects that beliefs 
usually come in degrees. Our beliefs are often more or 
less uncertain. For instance, you think that glyphosate 
probably kills bees, but you are not certain. We have a 
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very rich vocabulary for describing this quantitative level; 
we speak of probability, plausibility, believability, uncer- 
tainty, credibility, likeliness, and so on. In rationality 
research, the dominant quantitative account by far is 
Bayesianism, the doctrine that degrees of uncertainty or 
belief are subjective probabilities. This holds for psychol- 
ogy just as much as for philosophy (Jeffrey, 1992; Oaks- 
ford & Chater, 2007). However, one should observe that 
the label “Bayesianism” is only legitimate when these 
degrees conform to the axioms of mathematical probabil- 
ity theory. 

Probabilistic thinking has evolved for about 350 years 
(cf. Hacking, 1975).'’ It has pervaded most scientific dis- 
ciplines (Kriiger, Daston, & Heidelberger, 1987; Kriiger, 
Gigerenzer, & Morgan, 1987; for a philosophical intro- 
duction, see Gillies, 2000), and so it comes as little 
surprise that Bayesianism has become a very strong para- 
digm in psychology and philosophy as well. For this rea- 
son, we devote the entire section 4 of the handbook to 
it. From the Bayesian perspective, the qualitative charac- 
terization of belief states looks hopelessly vague, if not 
useless. Bayesianism has thus developed imperialistic 
tendencies, claiming that it is the only basic characteriza- 
tion of epistemic attitudes. If so, this would bring laud- 
able unification to the field. However, this imperialism 
has come under pressure from two sides. 

One reason is that one cannot simply discard the 
qualitative level, as advocated by the “radical probabi- 
lism” of Jeffrey (1992). The qualitative notion of belief 
seems too deeply entrenched in everyday discourse. So, 
the only option is to somehow reduce the one to the 
other level. However, there is no good reductive account. 
How weak may a belief be and still count as a belief? Can 
we accept the so-called Lockean thesis that something is 
believed if and only if the degree of belief in it is above a 
certain vague and perhaps contextually given threshold, 
just as we might say that a man is tall within the entire 
population if his size is above a certain threshold, say 
6'3", and tall among basketball players if he is above 7 
feet? In fact, the relation between belief and probabili- 
ties is currently the matter of many controversies.'® 

The second reason why it is not cogent to repre- 
sent degrees of belief as probabilities is that the past 
50 years have seen a plethora of alternative proposals 
(some of which have earlier precursors). An important 
driving force was artificial intelligence, which needed 
algorithms for uncertain reasoning, found probability 
theory infeasible, and hence looked for alternatives. In 
the meantime, Markov and Bayes net theorizing have 
much improved the computational manageability of 
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probabilities (Pearl, 1988). However, alternatives are avail- 
able (Halpern, 2003). The need was also raised from the 
psychological and the economic side, which found 
subjective probabilities descriptively wanting. Even the 
theory of mental models, which for a long time was 
dominated by binary classical logic, has more recently 
developed approaches for how to deal with degrees of 
belief without using Bayesian probabilities (Hinterecker, 
Knauff, & Johnson-Laird, 2016; Johnson-Laird et al., 
2015; Johnson-Laird & Ragni, 2019). This increases jus- 
tificational pressure on the normative side—why this 
rather than that mathematical format?—as well as on 
the descriptive side—why model human uncertainty in 
this rather than in that way? Discussions of such alterna- 
tives may be found in chapter 4.7 by Dubois and Prade; 
chapter 5.3 by Kern-Isberner, Skovgaard-Olsen, and 
Spohn; chapter 8.3 by Glockner; and chapter 8.4 by Hill 
(all in this handbook). 

Between the qualitative and the quantitative, there 
is, moreover, the comparative level, where one proposi- 
tion is taken as more plausible or credible or certain than 
another. For instance, you may take it to be more cred- 
ible that glyphosate harms bees than that sugar does. 
One may even take the comparative level as basic, since 
it also allows one to state the threshold idea of quali- 
tative belief and certainly underlies any quantitative 
measure of uncertainty (like one may base judgments 
of tallness as well as measurements of height on the 
relation “taller than”). Indeed, one may wonder which 
formal properties of those plausibility comparisons lead 
to which numerical measure. Again, this opens a large 
space of normative dispute as well as issues of empirical 
adequacy. In Knauff (2013), a comparative level has been 
developed for beliefs about spatial relations, which also 
plays a role in chapter 13.2 by Ragni and chapter 13.3 by 
Knauff (both in this handbook). However, this compara- 
tive level is not further developed in this handbook. 

So far, our remarks about how to conceive of epis- 
temic states were made only from a static or synchronic 
perspective. They have dealt with how epistemic states 
are at a given moment. This is basic—and only prepara- 
tory for taking a dynamic or diachronic perspective. It is 
important, we think, to distinguish two different aspects 
of the epistemic dynamics, internally and externally 
driven dynamics. Both are subject to considerations of 
empirical and normative adequacy, and they lead to dif- 
ferent kinds of dynamic issues, as we will describe now. 

There is, first, an internally driven dynamics, which 
is, roughly, brought about by thinking. It runs on the 
process-oriented (or “algorithmic”) level. As indicated at 
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the end of subsection 2.2 of this introductory chapter, 
this is a crucial issue for cognitive scientists but also for 
logicians. It is one thing to analyze the formal structure 
of what a reasoner receives as input—typically the set of 
premises and their formal structure—and then to look at 
the output the reasoner generates—typically a conclu- 
sion that is evaluated as (more or less) justified accord- 
ing to some normative standard. However, the history 
of psychology shows that approaches occupied only 
with input-output associations have limited explana- 
tory and predictive power. It is another and much more 
demanding thing to understand how the input to the 
human cognitive system is processed and why this leads 
to the observed output, for instance, a particular conclu- 
sion or belief. What mental machinery, involving which 
cognitive operations, lies behind these internally driven 
dynamics of human rationality? Here, the main ques- 
tions are: On the basis of which mental processes are 
people competent to reason rationally? Why are these 
processes sometimes error-prone? And how do reasoning 
processes interact with other cognitive and noncogni- 
tive psychological mechanisms? This is a more dynamic 
description of the main questions that have occupied 
the psychology of reasoning since its very beginning 
(Wilkins, 1928; Woodworth & Sells, 1935). The most 
influential theories from the psychology of reasoning 
are sketched in sections 2 and 4 of this handbook, which 
compare normative and descriptive theories of rational- 
ity in reasoning. 

The logical side also deals with a multitude of issues. 
There are not only actual but also many potential rea- 
soning processes, which may or may not be followed 
and which may or may not be correct (as is presupposed 
by speaking of errors). For instance, logic is full of differ- 
ent (but equivalent) sound calculi or proof procedures, 
whether or not they are actually used. In a qualita- 
tive picture, reasoning is, above all, logical inference. 
Piaget would have said the same about psychology. It 
is amply clear, though, that logical inference must not 
be restricted to deductive or monotonic inference. The 
logical zoo has become quite diversified, and there is a 
growing field of nonmonotonic logic, default logic, and 
defeasible reasoning. This field is partially represented in 
sections 5 and 6 of this handbook. Each logic comes with 
its own sound procedures or derivation rules and possi- 
bly with a semantic justification. And each of them may 
or may not be adequate for describing how the human 
mind works, and may or may not be approximated by 
mental models or heuristics. In a quantitative picture, 
reasoning is first and foremost probabilistic reasoning. 
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Again, though, one must emphasize that all the other 
quantitative formats come with their own accounts of 
reasoning. Again, this opens a rich field of normative 
and empirical assessment. 

However, this does not exhaust the dynamic perspec- 
tive. There is, second, an externally driven dynamics. This 
is about how epistemic states change under the influ- 
ence of external input: experience, perception, learn- 
ing, or information in general. Again, one may say that 
this is about drawing inferences from this input. Cog- 
nitive psychologists have developed several descriptive 
theories of how people account for new information, 
how they detect and resolve inconsistencies with prior 
beliefs, how they take into account the order in which 
new information comes in, how this affects the person’s 
entire set of beliefs, how they consider the trustworthi- 
ness and reliability of the source of new information, 
and how people deal with the fact that new information 
is inconsistent with their prior beliefs (Elio & Pelletier, 
1997; Johnson-Laird, Girotto, & Legrenzi, 2004; Politzer 
& Carles, 2001; Revlin, Cate, & Rouss, 2001). Such theo- 
ries are either motivated by approaches from artificial 
intelligence and philosophy, for example, principles of 
minimal change or epistemic entrenchment (Alchour- 
ron, Gardenfors, & Makinson, 1985; Gardenfors, 1984, 
1992), or they have genuine roots in psychology, for 
instance, when people consider the trustworthiness of 
information sources or try to explain conflicts between 
prior beliefs and new facts (Khemlani & Johnson-Laird, 
2011; Wolf, Rieger, & Knauff, 2012). 

While these process-oriented theories seek to recon- 
struct how the cognitive system deals with new infor- 
mation, we can also take an output-oriented perspective 
(Marr’s computational perspective) on the externally 
driven dynamics of belief. This is not about how the 
input is processed, but about how the goals of com- 
putation change through the input—they have their 
dynamics as well. Put plainly, it is about what to believe 
upon receiving some data and not about how to arrive 
at it. According to Spohn (2012, chapter 1), this prob- 
lem of revision is tantamount to the venerable problem 
of induction. Above, we mentioned the limitations of 
looking only at the input-output associations. However, 
this must not distract from the fact that these associa- 
tions have their own dynamics, which is not treated by 
focusing on the internally driven dynamics but rather 
needs to be studied on its own. The traditional proba- 
bilistic account of this issue consists in learning by con- 
ditionalization or, equivalently, by Bayes’ theorem; it 
has by now been considerably refined. However, each 
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epistemic format comes with its own learning theory. 
Philosophers tend to emphasize the externally driven 
dynamics; it is addressed in chapter 4.1 by Hajek and 
Staffel, chapter 4.2 by Hartmann, chapter 5.2 by Rott, 
and chapter 5.3 by Kern-Isberner, Skovgaard-Olsen, and 
Spohn (all in this handbook). 

Of course, the opposition between internally and 
externally driven dynamics is not as clear-cut as it may 
seem. For instance, suppositional reasoning also belongs 
to the internally driven dynamics, because a person does 
not require external input in order to work out what she 
would believe given a certain supposition. Then, how- 
ever, there seems to be a close connection between inter- 
nally and externally driven dynamics. Isn’t what we 
accept under a given supposition precisely that which 
we would accept after getting the information (through 
whatever channels) that the supposition actually holds? 
This is a topic that has been intensely discussed under 
the heading “supposing vs. updating” (see Spohn, 2012, 
chapter 9; Zhao et al., 2012), but we cannot go into the 
details here. We just want to emphasize once more that 
all theories of the dynamic aspects of rationality can be 
assessed in a normative and a descriptive dimension. We 
return to this in section 6 of this introductory chapter. 


3.2 Practical Rationality 

We have quite extensively discussed theoretical rational- 
ity. Many points, however, apply not to theoretical ratio- 
nality specifically but similarly to practical rationality, 
that is, to the rationality of actions or intentions and 
motivational attitudes in general. Recall that practical 
rationality presupposes and thus includes theoretical 
rationality. For this reason, we can carry over most of the 
conceptual distinctions we have already made, and we 
can be much briefer even though the topic is wider. The 
topic is wider because we now deal with a more com- 
prehensive conceptual field. We now deal not only with 
epistemic attitudes but also with decision making and 
motivational matters, which we have already unfolded 
in subsection 2.2 of this introductory chapter. 

A first issue of practical rationality is whether it is 
legitimate to lump together drives, wishes, motivations, 
wants, goals, aims, norms and values, etc., as the topics of 
practical rationality. In a theoretical spirit, many philos- 
ophers tend to do so by simply speaking of desires, and 
many economists do so by measuring all of them on a 
single utility scale. Basically, they treat drives and motiva- 
tions in the same manner as other-regarding preferences 
and moral obligations. However, other philosophers, 
economists, and psychologists emphasize the differences 
between the various kinds of such pro-attitudes. We do not 
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want to take a stance here, because in the end, it is a theo- 
retical issue whether or not the various kinds of desires 
work according to different theories. Most psychologists 
would probably argue for such a separation. Perhaps the 
“imperialism” of utility theory here is as problematic as 
that of Bayesianism in theoretical rationality. Still, here 
we ignore these potential differences, as our conceptual 
points about desires presumably apply across the board. 

So, let us focus on the same questions as we already 
did on the side of epistemic rationality. The first ques- 
tion is again which entities those desires are about, that 
is, what it is that we want, desire, etc. On the epistemic 
side, we already mentioned some problems regarding 
propositions as the contents of beliefs. Now, in prac- 
tical rationality, the issue is even more obscure with 
respect to desires and motivational attitudes. If we treat 
the contents of desires as propositions, this might not 
fully capture our ordinary talk. We may also say that the 
entities we desire are objects or goods or actions, and 
so on. There is perhaps a unification: “I want this bike” 
is the same as “TI desire that I possess this bike,” and “T 
intend to do a” is the same as “I intend that I perform 
action a” (Jeffrey, 1965, chapter 4). One may think that 
objects, goods, and actions are clearer contents of desires 
than propositions. But this is not true. For instance, I 
love Wonder Woman, but she does not actually exist. So, 
what is the object of my desire here? Another example: I 
may want to meet Dr. Jekyll but not Mr. Hyde. This is not 
possible, because both are the same person. There are no 
two different objects here that I could desire to meet or 
not to meet. Philosophers then tend to say that desires 
are about intentional objects or about objects under a 
description. And they prefer to consider the contents 
of desires to be propositions because propositions can 
contain objects in such a modified understanding.'” We 
are aware that this may sound weird to people who are 
not usually involved in such theoretical considerations. 
Hence, we better avoid delving into those problems and 
simply stick to propositions, for ease of exposition. 

The next distinction that we made within the realm 
of theoretical rationality was the one between qualita- 
tive, quantitative, and comparative shapes of mental 
states. We have the same distinction on the side of prac- 
tical rationality. In a qualitative framework, propositions 
can simply be or not be desired, wanted, mandatory, or 
the like. Recall, though, that we need to account for the 
interaction between desires and beliefs or epistemic atti- 
tudes in general, because we must respect the basic dis- 
tinction between intrinsic and extrinsic desires, which 
we introduced in subsection 2.2 of this introductory 
chapter. Extrinsic desires involve such interaction, since 
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they are directed at propositions somehow believed to be 
conducive to intrinsically desired propositions. Within a 
purely qualitative framework, however, it is very diffi- 
cult to account for this interaction (for a proposal and 
further references, see Spohn, 2020). 

So, just as on the epistemic side, where we talked 
about degrees of belief, we should allow desires, too, to 
come in varying strength. If we do so, we again have 
two options. We can either express the strengths of 
desires quantitatively, or we can express them merely 
on a comparative level. In fact, the comparative level is 
more prominent on the practical than on the epistemic 
side. Thus, we enter the large field of preference theory. 
The relation between the comparative and the quali- 
tative level has been intensely studied in the theory of 
revealed preferences (Samuelson, 1938), which originally 
served as a way of operationalizing preferences by means 
of choice behavior. The comparative and the quantita- 
tive level are also closely tied together, namely, by the 
famous von Neumann-Morgenstern utility theory (von 
Neumann & Morgenstern, 1944, chapter I.3), which 
showed how we can measure utility functions on the 
basis of preferences in a sufficiently unique way. Before 
this theory, numerical strengths of desires seemed to be 
elusive things of doubtful scientific status, but afterward, 
talk of numerical utilities seemed legitimate (for all this, 
see chapter 8.1 by Griine-Yanoff, this handbook). 

Thus, talk of utility functions measuring motiva- 
tional strengths by real numbers is by now well estab- 
lished and even dominant, at least in economics and 
philosophy. The great advantage of utility functions is 
that they combine so smoothly with probability mea- 
sures, the prevalent numerical representation on the 
epistemic side. This combination culminates in the 
principle of maximizing expected utility, the fundamen- 
tal rationality principle of standard decision theory (see 
chapter 8.2 by Peterson, this handbook). No wonder 
there is a tremendous debate about the normative foun- 
dations of this quantitative framework and its empirical 
adequacy. In the past decades, three cognitive and social 
scientists—Herbert Simon, Daniel Kahneman, and Rich- 
ard Thaler—were awarded the Nobel Prize in Econom- 
ics for showing how and explaining why people deviate 
from these norms of standard decision theory. This stim- 
ulated a number of alternative accounts, which are rep- 
resented in chapter 8.3 by Gléckner, chapter 8.4 by Hill, 
and chapter 9.4 by Dhami and al-Nowaihi (all in this 
handbook). Today, this field is quite variegated and het- 
erogeneous. It is sometimes called “qualitative decision 
theory,” where “qualitative” does not have the mean- 
ing from above, but only signals that the field moves 
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to a less fine-grained way of description than offered by 
probabilities and utilities, although it does not belong to 
the qualitative level as we use the term here. 

Let us now come to the next distinction we made on 
the side of epistemic rationality. Is there also an internally 
and an externally driven dynamics of desires, wishes, moti- 
vations, and so on? Yes, and in principle we can make 
the same distinctions, although the algorithms of prac- 
tical reasoning certainly differ from those of theoretical 
reasoning. On the side of the internally driven dynam- 
ics, it is tempting to interpret decision theory as an algo- 
rithmic theory about practical reasoning processes at the 
process-oriented level. Decision theory provides gen- 
eral rules for calculating expected utilities and thus for 
determining which options maximize expected utility.”° 
However, thus interpreted, decision theory seems descrip- 
tively inadequate—people apparently do not follow these 
rules and do not do any of these calculations. Hence, we 
may prefer to locate decision theory on what we call the 
output-oriented level. In the attempt to be more adequate 
on the process-oriented level, cognitive psychologists 
have developed various accounts of so-called fast and fru- 
gal heuristics and bounded rationality (see chapter 8.5 by 
Hertwig & Kozyreva, this handbook). However, the goal 
of developing psychologically plausible algorithmic theo- 
ries of practical rationality is still daunting, probably even 
more demanding than in the area of epistemic rationality. 

What about the externally driven dynamics? In prac- 
tical rationality, this becomes relevant as soon as we 
conceive decision theory not as a process-oriented but as 
an output-oriented theory that only states which inten- 
tion should result from the given utilities and probabili- 
ties. Here again, epistemic rationality comes into play. 
Desires and intentions are also affected by the exter- 
nally driven dynamics of epistemic states. We learn— 
and thereby change our extrinsic desires and indeed our 
intentions. This is a very important point, which many 
people forget when talking about desires and practical 
rationality: we often wait for, or indeed seek, informa- 
tion, and depending on what we learn, we intend to do 
either this or that. For instance, once I have learned the 
weather forecast for this afternoon, I may change my 
desire to go to the demonstration in a T-shirt. There are 
sophisticated theories elaborating on strategic rationality, 
that is, on plans or strategies optimally responding to 
various possible pieces of information (see Raiffa, 1968). 

So far, we have only discussed the dynamics of extrin- 
sic desires or intentions. Is there also an externally driven 
dynamics of intrinsic motivational states? Sure. Psy- 
chologists have extensively studied this, although not 
under the heading of rationality. In motivation research, 
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our actions are seen as either driven by relatively stable 
behavioral dispositions or triggered by stimuli from the 
environment, for instance, when you see appetizing food 
or a physically attractive person (Heckhausen & Heck- 
hausen, 2018). However, even the relatively stable dispo- 
sitions are not immutable. Our pro-attitudes change and 
evolve all the time, due to all kinds of influences apart 
from information. Their dynamics is also driven by edu- 
cation in general and moral education in particular, by 
the social environment, by fashion and advertising, by 
saturation, boredom, and curiosity, simply by maturing 
and aging, etc. So, this is usually considered an issue of 
descriptive theories of rationality. 

However, the dynamics of desires is not only a fac- 
tual matter to be studied empirically. It also raises nor- 
mative issues. And many of them are of a moral kind. 
We want people to acquire the right values and try to 
prevent them from adopting wrong ones, whatever they 
are. These topics fall outside the scope of this handbook, 
since they are about morality, not rationality. 

Another field that is not represented in this handbook, 
although it has to do with externally driven dynamics of 
desires, is dynamic decision theory (McClennen, 1990). It 
deals with what economists call “endogenous preference 
change” (Bowles, 1998; Loewenstein & Elster, 1992), that 
is, precisely with our present issue. The question there is 
not which changes would be rational but how to behave 
rationally in view of preferences changing due to what- 
ever influences.’ Thus, the change itself need not be 
assessed as rational, but when deciding, we may take such 
changes into account in a rational way. However, such 
considerations seem to require optimizing according to 
two different points of view, the old preferences and the 
new preferences. From the perspective of standard deci- 
sion theory, which is about optimizing within only one 
point of view, this is a difficult, if not unsolvable, ques- 
tion. So, this field has thus far remained esoteric and will 
be neglected here. 


3.3. Truth and Rationality 
At the beginning of this section, we argued that it is 
important to distinguish between theoretical and practi- 
cal rationality—a distinction that roughly matches the 
distinction between reasoning and decision making in 
psychology. But then we saw that these two types of 
rationality have many things in common and that theo- 
retical rationality is a precondition of, and thus part of, 
practical rationality. So, why still sharply distinguish the 
two types of rationality? 

The answer seems simple: because epistemic rational- 
ity is about the rational justification of beliefs, whereas 
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practical rationality is about the rationality of means 
to an end. But this is too simple. A crucial point is that 
epistemic rationality is not merely about the justification 
of beliefs. Things are a bit more complicated because we 
need to distinguish between epistemic and nonepistemic 
justifications of beliefs. A standard example, slightly mod- 
ified, comes from BonJour (1985, p. 6): a mother ought 
to, and does, believe in the innocence of her son even in 
the face of overwhelming evidence that he committed a 
severe crime. One may take it to be morally required that 
at least the mother backs her son, and given this moral 
requirement, she may be rationally justified in having 
that belief. However, given the evidence, the mother 
would be epistemically irrational to believe in her son’s 
innocence. This shows that the assessment of beliefs as 
rationally justified may involve more criteria than merely 
epistemic justification. The justification of beliefs may 
also take nonepistemic and specifically moral forms. This 
is not per se a reason to call such beliefs irrational. They 
are, however, epistemically irrational. Theoretical ratio- 
nality is restricted to the epistemic part of justification. 

But then the question arises whether nonepistemic 
justifications of beliefs are still acceptable. We agree 
with BonJour (1985) and many other philosophers that 
the rational formation of beliefs and doxastic states is 
subject only to the norms of epistemic rationality. This 
also means endorsing the normative principle that any 
external considerations demanding epistemic irrational- 
ity must be rejected. It was a long historic fight to estab- 
lish this principle, starting with the progress from myth 
(mythos) to reason (logos) in Ancient Greek philosophy, 
and continuing with the rise of scientific methods, with 
Cartesian methodical doubt (which called into question 
any dogmatic truth), with Kant’s characterization of 
Enlightenment: “Have courage to use your own mind!” 
or with the claim of Peirce (1877) that the scientific 
method is the only appropriate one for fixing our beliefs. 
Certainly, the fight is not over. 

The task, then, is to more substantially describe this 
narrow sense of epistemic rationality. Philosophers dis- 
cuss various specific aims of our epistemic activities, and, 
as mentioned at the beginning of this section, some pro- 
ceed to formulate a specific epistemic decision theory. 
Among these aims may be simplicity, systematicity, rel- 
evance, explanatory power, and perhaps even aesthetic 
values. For sure, though, the primary aim is truth. This 
first maxim was stated by William James (1896/1956, 
section VII) in the shortest possible way: “Believe truth! 
Shun error!” Some philosophers prefer to identify knowl- 
edge instead of merely truth as the aim of belief (see 
Chan, 2013; Williamson, 2000, chapter 11). However, in 
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subsection 2.2 of this introductory chapter, we already 
decided to leave the difference between knowledge and 
true belief undiscussed. 

Of course, belief formation does no more than aim 
at truth. Rational belief formation in no way guarantees 
truth. This is part of our skeptical heritage. We may acci- 
dentally hit the truth in irrational ways. And we may 
arrive at false beliefs in a perfectly rational way, not only 
individually but also collectively. In principle, the pos- 
sibility of misleading evidence and false theorizing can 
never be excluded. This is why philosophers say that the 
reasons we have for our beliefs are no more than truth- 
conducive. The point of the normative principle stated 
above is only that there is no alternative way to approach 
the truth. 

It is a difficult issue whether all other epistemic goals 
can be reduced to the aim of truth or knowledge. Willard 
Van Orman Quine once famously claimed that “norma- 
tive epistemology is a branch of engineering,” namely, 
“the technology of truth-seeking.” “The normative here, 
as elsewhere in engineering, becomes descriptive when 
the terminal parameter is expressed” (Quine, 1986, 
pp. 663-664). We doubt this (see Spohn, 2012, chap- 
ter 1) but need not expand on the issue now (see also 
subsection 4.3 of this introductory chapter). Moreover, 
it is quite unclear how to transfer the aim of truth to the 
other epistemic formats mentioned above. For instance, 
it makes no sense to call subjective probabilities true or 
false. A key notion discussed instead is that probabili- 
ties may be justified by their greater or lesser accuracy 
(see chapter 4.1 by Hajek & Staffel, this handbook). In 
fact, this point speaks in favor of the indispensability of 
the notion of belief, because beliefs can plainly be called 
true or false. However, these problems cannot distract 
from the fact or the norm that our epistemic activities 
essentially aim at truth. 

But then the question is: what is truth? Aristotle’s 
correspondence theory is the traditional paradigm. 
It has been amended by many modern versions: Tar- 
ski’s semantic theory, deflationary and disquotational 
truth theories, the redundancy theory, and so on (see 
Kirkham, 1992; Kiinne, 2003). All these theories are 
epistemologically not enlightening and do not intend 
to be so. Because of this deficit, a host of further truth 
theories have been proposed: Peirce’s and James’s prag- 
matic theory, Habermas’s consensus theory, idealistic 
and constructivist truth theories in various forms, the 
coherence theory of truth, again in various forms, and 
so forth.” Already the labels indicate that we are movy- 
ing into a very controversial terrain that we cannot dis- 
cuss here. 
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The main difficulty is that psychologists and philos- 
ophers tend to have quite different opinions and theo- 
ries about the notion of truth. On the one hand, the 
opinions of most psychologists are shaped by countless 
experimental findings showing how deeply people’s 
understanding of the world is mediated by their experi- 
ence and constructed by their brains. This may suggest 
that the idea that we perceive the “real world” is just an 
illusion, a concern that can be traced back to the work 
of Piaget, the learning theory of Bruner (1957), and the 
social constructivism of Vygotsky (1978).” The skepti- 
cism goes even further back to the groundbreaking work 
of Sir Frederic Bartlett, who already emphasized the 
reconstructive character of the human mind and the 
social factors that play an important role in what we 
think is true of the world (Bartlett, 1932/1995). 

On the other hand, philosophers could say that such 
findings about the constructive achievements of the 
brain inform us about how people come to take some- 
thing to be true, but not about truth itself; they may 
find the empirical views of psychologists only indirectly 
relevant to the multitude of philosophical truth theories 
mentioned above. However, it is important for philoso- 
phers that the notion of truth is distinct from, and does 
not collapse into, the notion of belief (of taking some- 
thing to be true). This leads to an objective or at least 
intersubjective notion of truth, which most psycholo- 
gists might accept as an ideal but often question on 
empirical grounds. 

Maybe psychologists and philosophers should try to 
collaborate more on the topics of truth and objectiv- 
ity. This is also important for the distinction between 
practical and epistemic rationality. If beliefs aim at 
truth, as said above, and do so in some not entirely 
subjective sense, then epistemic rationality serves a dis- 
tinguished, not entirely subjective, aim. By contrast, it 
is very doubtful that there are corresponding aims on 
the side of practical rationality. We rationally pursue 
our practical goals, which may be many. And we all 
should behave morally. However, whether there is an 
(objective) moral truth is the highly contested issue of 
moral realism, which is still undecided (see, e.g., Sayre- 
McCord, 1988; Schroeder, 2018). And even if there is 
something like moral truth, it is highly contested in 
turn whether the pursuit of this moral truth is a demand 
of rationality (see also chapter 12.1 by Fehige & Wessels, 
this handbook). By contrast, the pursuit of truth might 
be the key issue of epistemic rationality. This is what 
sets apart the two domains and makes epistemic ratio- 
nality special. 
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4. Normative and Descriptive Theories of Rationality 


Whenever we talk about theoretical and practical ratio- 
nality, about beliefs, knowledge, desires, pro-attitudes, 
etc., this has a normative and a descriptive dimension. 
The distinction is at the heart of our philosophico- 
psychological enterprise and was already implicitly used 
in the previous sections. The core of the distinction 
can be made explicit in three questions: We ought to be 
rational, but what exactly does this require from us? We 
think we are rational (at least to some extent), but how 
do we actually reason? And what is the relation between 
these two questions? 

The first question is more at home in philosophy. It 
is concerned with the normative standards of rationality. 
The second question has been at the center of psycho- 
logical research since the very beginning of the disci- 
pline. It is concerned with the development of descriptive 
theories of human rationality. The third question might 
be the most controversial. It is motivated by the seem- 
ingly huge gap between normative and descriptive 
approaches to human rationality: people seem to com- 
mit many errors and suffer from biases in their thinking, 
reasoning, and decision making, when measured against 
standard normative systems such as classical logic, prob- 
ability theory, or decision theory. Our aim in this section 
is to illustrate the normative-descriptive distinction by 
explaining some central issues and results on both sides. 
We also want to discuss the relation between the two 
perspectives, where they diverge, where they converge, 
and how we conceive of this relation, to obtain a com- 
prehensive theoretical and empirical understanding of 
human rationality. 

We should start by clarifying what we mean when 
we speak of “ought” or “norms.” These terms are sys- 
tematically ambiguous. We may speak about empirical 
or about genuine normativity. Empirical normativity 
concerns the norms we empirically find in a certain 
community. For instance, when we come to Great Brit- 
ain, we learn that one ought to drive on the left; this is 
an empirical fact about Great Britain. Another example 
is religion. We can empirically study which religious 
norms are accepted in certain communities and thus 
mostly followed. In a genuinely normative perspec- 
tive, by contrast, the empirical norms can never settle 
whether we should really drive on the left or whether 
we ought to follow certain religious norms. We cannot 
find out merely by empirical investigation what ought 
to be the case and what we genuinely ought to do or 
believe. It is a matter of normative deliberation and of 
taking or accepting a normative stance. In this sense, it 
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is basically a fallacy to derive ought from is (and also is 
from ought). 

This is an important distinction, which is often blurred. 
Within legal philosophy, it has been strongly emphasized 
by Hart (1961, pp. 54ff.), who distinguished normativity 
viewed from an external and from an internal perspective. 
From the external perspective, it is just an empirical ques- 
tion what is normative in a given group (“In the UK, they 
recognize the law to drive on the left ...”). In this per- 
spective, there is no opposition between the descriptive 
and the normative. An outside observer can recognize an 
empirical norm to hold just by noticing that it is mostly 
followed and that deviations are sanctioned by law or by 
the fellow people. In the internal perspective, by contrast, 
we ourselves accept norms as standards governing how 
we ought to act. Only in this internal perspective do we 
have a real distinction between normative and descrip- 
tive theories of rationality. It is this real distinction that 
we want to discuss in this section. Here, normativity is 
always understood in the genuine or internal sense. 


4.1 Normative Theories 

Genuine normativity is a huge topic for philosophers. 
But, we argue, it is also highly relevant for psychol- 
ogy: empirical research needs some normative reference 
points as benchmarks for human thinking and action. 
This is sometimes denied by cognitive scientists (Elqa- 
yam & Evans, 2011; Elqayam & Over, 2016), a position 
that we will discuss in subsection 4.3 of this introduc- 
tory chapter. However, psychological research is full of 
examples where empirical findings are compared to nor- 
mative standards: memories are compared with actual 
events, responses to visual stimuli are classified as “hits” 
or “misses,” spoken sentences are evaluated as syntacti- 
cally correct or incorrect, and even emotional states are 
judged as either “normal” or pathological, to name just 
a few examples. In rationality research, normative stan- 
dards usually come from logic, probability theory, and 
decision theory. So, let us start by discussing these theo- 
ries, although we shall see that the normative discussion 
is much broader. 

Let us first look at the norms of theoretical rationality. 
Already classical logic is a complicated case. If we follow 
Frege, logic is not a normative theory at all. It states the 
abstract laws of truth, which are what they are indepen- 
dently of any thinker. Logical truth is like mathematical 
truth. 2+3 = 5—this is an atemporal mathematical truth. 
Logic becomes normative only when we add that the laws 
of truth are at the same time the laws of correct thinking, 
that our reasoning ought to follow the deductive rules of 
logic (see also chapter 3.1 by Steinberger, this handbook). 
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Indeed, this seems to go without saying. Didn’t we say 
that truth is the central aim of rational belief? Now, the 
crucial feature of logical deduction is truth preservation: 
it inevitably carries us from true premises—if they are 
true—to true conclusions. This is the standard argument 
for the epistemic value of classical logic. But it does not 
necessarily speak in favor of classical logic. It could also 
speak for other systems of logic, such as intuitionistic, rel- 
evance, or paraconsistent logic. 

Moreover, the fact that logic is truth-preserving 
does not imply that only logical deduction is rationally 
legitimate. Recall James’s maxim quoted above: “Believe 
truth! Shun error!” Of course, logic is perfect for the sec- 
ond aim, shunning error: if we do not start from an error, 
logic does not introduce one. However, concerning the 
first aim, believing truth, logic does very poorly. It does 
not tell us what to believe, apart from the logical conse- 
quences of the given information. In other words, what 
is not implicit in the premises never becomes believed; 
logic is not ampliative. Obviously, we need more than 
logic. We shall return to this point. 

Another point is that classical logic cannot handle 
degrees of belief. Yet we need a normative theory that 
tells us how we ought to deal with uncertainty. And there, 
probabilities are the natural choice. Again, though, prob- 
ability theory is just a mathematical theory to be used 
for many and variegated purposes and under different 
interpretations. One of them is to interpret probabilities 
as subjective degrees of belief, and then it is obviously a 
normative requirement that these degrees should behave 
like mathematical probabilities. Indeed, philosophers 
have devised a plethora of justifications for considering 
degrees of belief as probabilities, the most important of 
which are represented in chapter 4.1 by Hajek and Staffel 
(this handbook). One type of justification, accuracy argu- 
ments, even attempts to justify probabilities in terms of 
truth approximation (=accuracy), that is, with reference 
to truth as the epistemic goal (Joyce, 2009; Pettigrew, 
2016). These justifications extend to learning rules, that 
is, rules for how to revise one’s degrees of belief after 
receiving new information or evidence. Bayes’ theorem, 
which is tantamount to the rule of conditionalization, is 
the main rule. There are, however, various other learning 
rules studied in the literature (see again chapter 4.1 by 
Hajek & Staffel, this handbook). 

Note that this is a purely normative discussion. If there 
should be other normatively defensible conceptions of 
degrees of belief, the entire argument cannot be cogent. 
Indeed, some alternative conceptions of degrees of belief 
will be mentioned below. It is therefore unfortunate that 
psychologists widely take the normative requirements 
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of subjective probability theory for granted and thus 
focus only on empirical evidence for, or counterevidence 
against, this assumption. 

It should also be noted that the requirement to obey 
the mathematical axioms of probability is still very weak. 
It is, for instance, compatible with the strange anti- 
inductive inference to ever Jess expect the next swan to be 
white, the more white swans one has seen. So, epistemic 
rationality requires much more than these axioms, but 
it is highly contested what this might be (see, e.g., the 
attempts of Carnap, 1971, 1980, at an inductive logic, or 
D. Lewis, 1980; see also chapter 4.1 by Hajek & Staffel, 
this handbook). 

Let us now turn to the norms of practical rational- 
ity. For a long time, this field was dominated by deci- 
sion theory. This is, however, as problematic as the use 
of classical logic and Bayesian probability for epistemic 
rationality. The standard decision theory has been codi- 
fied by Savage (1954). Its central principle of maximiz- 
ing expected utility, often called “Bayes’ principle” (not 
to be confused with Bayes’ theorem), is primarily a nor- 
mative principle. Again, some psychologists seem to 
see it as the only normative option and then raise their 
empirical objections. Economists and philosophers, by 
contrast, have devoted much effort to justifying the 
standard theory. The main argument is: If your prefer- 
ences have certain normatively commendable features, 
then your utilities can be measured on an interval scale 
such that your most preferred option is one that maxi- 
mizes expected utility. Savage (1954) was the first to 
extend this kind of argument to probabilities and utili- 
ties simultaneously (see also chapter 8.2 by Peterson, 
this handbook). Indeed, probability and utility theory 
support each other, and neither is easily replaced by 
some alternative in the presence of the other. 

There are many empirical criticisms of standard 
decision theory, as we will see below. However, there is 
also considerable normative criticism, as illustrated, for 
example, by Ellsberg’s paradox (Ellsberg, 1961). There is 
an urn before you with 90 balls of various colors. More 
precisely, it contains 30 red balls and 60 other balls. The 
60 other balls are either black or white in an unknown 
ratio. You have to guess the color of the ball drawn next. 
If you are right, you get a reward; otherwise, you get 
nothing. Now, you first have a choice between guessing 
that the ball is red and guessing that it is white. Most 
subjects prefer to bet on red—the known risk regard- 
ing red seems preferable to the uncertainty regarding 
white. Second, you are offered a choice between guess- 
ing that the next ball is black-or-red and guessing that 
it is black-or-white. Here, most people prefer to bet on 
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black-or-white, perhaps because this time, black-or- 
white has a known risk while black-or-red has not. But 
this seems irrational—and is so according to standard 
decision theory. The first preference seems to take red 
as more likely than white, while the second preference 
seems to take red as less likely, which is inconsistent. 

The point of the example is not to show that people 
are somehow irrational. Rather, it intends to demon- 
strate that the allegedly inconsistent preferences are at 
least plausible and that there is something wrong with 
standard decision theory. Thus, we also find attempts 
to rationalize (not merely explain) these preferences, for 
instance, by introducing a nonadditive kind of probabil- 
ity with a corresponding alternative way of calculating 
expected utilities. Generally, such normative criticisms 
tend to generate alternative theories that can cope with 
the objections, and we could enter here a variegated 
landscape of normative disputes in decision theory (see 
chapter 8.4 by Hill, this handbook). 

So much about the difficulties with the norms most 
often used in psychological research. We should, how- 
ever, emphasize that the normative discussion about 
rationality is much richer and has produced many more 
proposals than we can report here. In fact, the entire 
handbook is supposed to give such an overview, but 
even this is incomplete. We have briefly mentioned an 
alternative to standard decision theory, and we have 
already announced that probability theory is by far not 
the only account of degrees of belief. Alternatives are 
plausibility theory, the Dempster-Shafer theory of belief 
functions, possibility theory, ranking theory, imprecise 
probabilities, and so on (see the overview in Halpern, 
2003). Some of these accounts are represented in this 
handbook (see chapter 4.7 by Dubois & Prade and chap- 
ter 5.3 by Kern-Isberner, Skovgaard-Olsen, & Spohn). 
Moreover, these alternative approaches often come with 
their own decision theory (see Halpern, 2003, chapter 5; 
Spohn, 2017, 2020). We mentioned that, with the excep- 
tion of chapter 8.1 by Griine-Yanoff, treating preference 
theory, the comparative level is neglected in this hand- 
book. However, this is an area of rich normative theo- 
rizing as well, concerning both practical and theoretical 
rationality. Certainly, cognitive research would benefit 
from more seriously considering such approaches, rather 
than limiting itself to the traditional reference points of 
logic, probability theory, and decision theory. 

The gap between the normative and the descriptive 
perspective is perhaps most apparent in the realm of logic. 
Above, we mentioned the strengths and the weaknesses 
of classical logic. It is obvious that most of our reason- 
ing is ampliative. This is not something to be criticized 
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but to be accounted for. The need to go beyond classical 
logic was perhaps most strongly felt concerning the con- 
ditional “if-then.” The long-known paradoxes of material 
implication” clearly showed that material implication as 
provided by propositional logic does not represent the 
conditional.” This point had dramatic effects. One may 
even say that the philosophy of logical empiricism failed 
due to the inability of classical logic to cope with the con- 
ditional.”° The situation changed only with the discovery 
of conditional logic by Stalnaker (1968),” which in turn 
was based on developments in modal logic about 10 years 
earlier. Since then, the field has exploded. 

Another strong influence came from AI in the 1970s, 
which also saw the need to go beyond classical logic in 
order to build ampliative inference into the computer. 
To that purpose, AI researchers developed various kinds 
of nonmonotonic, default, and circumscription logics. 
These developments are not represented in this hand- 
book (but see Gabbay, Hogger, & Robinson, 1994). How- 
ever, the field soon merged with philosophical attempts, 
and today a quite confusing multitude of nonmonotonic 
reasoning systems is available. This is partially a play 
with formal possibilities, but it is also a field of serious 
normative dispute. There are stronger and weaker sys- 
tems, and it is a normative issue whether the axioms and 
rules of these systems are acceptable. This field is incom- 
pletely represented in sections 5 to 7 of this handbook 
(but see also Koons, 2017). Again, we feel that this is a 
domain where the distance between psychologists and 
philosophers should be reduced. A recent attempt in this 
direction can be found in Ragni, Eichhorn, Bock, Kern- 
Isberner, and Tse (2017). The work of Stenning and van 
Lambalgen (2008) is another attempt that uses logic pro- 
gramming to model human nonmonotonic reasoning. 

Is there a common methodology behind all these 
ways of normative theorizing? Indeed there is, although 
as far as we know, it has received little systematic discus- 
sion. What one finds are somewhat vague approaches 
in which the methodology of normative theorizing is 
assimilated to the methodology of empirical theorizing. 
The idea of such approaches is not that normative theo- 
rizing would be in search of something like normative 
truth. This would be a problematic idea. Rather, it is in 
search of normative agreement, of the better and maybe 
decisive argument. Normative theories of rationality are 
just hypotheses about what is the best way to come to 
rationally justified beliefs or actions. 

This normative theorizing is much the same as cog- 
nitive or empirical theorizing in general. Typically, 
cognitive psychologists use data from controlled experi- 
ments and develop theories that agree with these data. 
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Normative theories also build on a kind of basic data, 
although not on empirical or experimental data, but 
rather on basic normative reference points, such as nor- 
mative intuitions or primitive normative assessments. 
For instance, imagine you know that, so far, I have seen 
very many white and very few black swans, and now 
you see me betting that the next swan will be black. 
Spontaneously and indignantly, you comment, “That’s 
silly!” This is a primitive assessment of epistemic ratio- 
nality, not yet backed by a theory, but very likely to be 
respected by any systematization of such assessments. 
Or imagine you give me a voucher for a movie ticket 
and I throw it away. You are upset and shout, “You are 
crazy!” This expresses an intuition of practical irratio- 
nality. Or you might reproach me: “You can’t say that 
we should host the refugees and at the same time vote 
for the nationalists!” Obviously, some of these primitive 
normative assessments are very strong, while others are 
weaker and fleeting: what at first looks unreasonable 
perhaps turns out after closer inspection not to be so. 

These primitive normative intuitions are not data in 
the sense typically used in science, but they are similar. 
Experimental data are the reference points for empirical 
theories: the data should be predicted and explained by 
the theory. Similarly, normative theories must agree with 
and justify those normative reference points or primitive 
assessments. So, from there on, normative theorizing pro- 
ceeds just like cognitive theorizing. We try to find rules of 
moderate generality, we think about first principles, and 
we systematize our normative assertions. Whenever there 
is an incoherence in our structure—for example, when a 
rule or principle has counterintuitive consequences—we 
try to solve the problem, and so forth. Perhaps the situ- 
ation is more fluid than in empirical theorizing, because 
our primitive normative assessments are not fixed and 
may sometimes change under the influence of convinc- 
ing theories. In the end, however, we are satisfied when 
we have brought everything into a good and stable reflec- 
tive equilibrium.” Yet this equilibrium is not a matter of 
taste; it is a matter of careful and ever more embracing 
argument, even though the result cannot be called “the 
truth.” We will return to this issue in subsection 4.3 of 
this introductory chapter. 


4.2 Descriptive Theories 

Let us now turn to descriptive theories of rationality. Devel- 
oping empirical theories about the cognitive processes 
underlying human rationality is arguably one of the most 
challenging endeavors of the cognitive sciences. Research 
from the past decades shows that people often draw 
unjustified conclusions from given premises, that is, they 
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do not adhere to the principles of theoretical rationality. 
Similarly, most people and organizations accept more 
losses for a potential high gain than the rules of rational 
choice theory seem to allow; national lotteries and equity 
trading are good examples. Hence, people deviate from 
the alleged norms of practical rationality. So, does the 
design of the human mind really provide the necessary 
cognitive abilities to solve the complex problems we are 
facing, as individuals and as society? What are the limi- 
tations of our rational capacities? Many chapters of this 
handbook seek to give answers to such questions; some 
are empirically robust and well founded, and others are 
more tentative and still under debate. 

Let us first look at the descriptive side of theoretical 
rationality. Almost all research in this area is concerned 
with reasoning, that is, people’s ability to draw ratio- 
nally justified conclusions from given premises. As we 
already said, the three main cognitive issues are compe- 
tence, errors, and the interaction of reasoning with prior 
knowledge and beliefs. 

Different descriptive theories explain human reason- 
ing competence. Some accounts assume that people reason 
epistemically by syntactic, language-based mental deriva- 
tions. On the output-oriented level, such accounts explore 
whether or not the output of the reasoning process is logi- 
cally justified by the input the reasoner received. On the 
process-oriented level, the key idea consists in a repertoire 
of inference rules represented in long-term memory. These 
rules are derived from general knowledge and refer to sen- 
tential connectives such as “if-then” and quantifiers like 
“all” and “some.” Reasoning is a process of transferring 
the inference rules into working memory and applying 
them to the given premises, which are also represented 
in a language-like format. The result is a language-based 
conclusion (e.g., Rips, 1994). Obviously, such accounts, 
in order to model reasoning processes, use logic as both 
a normative standard and a descriptive framework. The 
core idea is that human reasoning proceeds in analogy to 
the proofs of formal logic, although some logical connec- 
tives might be understood in a way diverging from propo- 
sitional logic. The natural-logic account in chapter 3.2 by 
O’Brien (this handbook) is an instance of this theoretical 
framework. Many versions of this account were inspired 
by the developments in AI since the 1960s, where most AI 
systems were so-called production systems and inspired 
by the idea of a general problem solver (Newell, 1973; 
Newell & Simon, 1972). 

Other descriptive theories assume that human rea- 
soning competence consists in processing subjective 
probabilities (e.g., Oaksford & Chater, 2007, 2020). We 
already mentioned this account in our list of milestones 
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in subsection 2.1 of this introductory chapter. Such the- 
ories primarily focus on the output-oriented level. They 
take the premises as input to the cognitive system and 
then compare the output to the alleged normative stan- 
dards of Bayesian probability calculus. Although most 
proponents of this account say that they are not inter- 
ested in explaining mental operations on the process- 
oriented level, it is clear that such accounts must rely 
on some kind of rules for the computation of subjec- 
tive probabilities. An important rule in this context is 
Bayes’ theorem (Oaksford & Chater, 2001, 2007). How- 
ever, since proponents of this account just formulate 
(output-oriented) computational-level theories, they do 
not explain how the cognitive system actually computes 
conclusions according to this theorem. 

Yet other theories adopt the position that people use 
mental simulations to draw epistemically rational infer- 
ences. On the output-oriented level, people use the input 
to create models of what would be the case if the input is 
true and produce the output as a result of a mental simu- 
lation performed to find new information not explicitly 
contained in the input. These mental models capture 
possibilities of how the world is, or could be, under 
certain conditions (e.g., Johnson-Laird, 2001, 2010; 
Johnson-Laird & Byrne, 1991). The key assumption on 
the process-oriented level is that reasoning does not rely 
on syntactic operations, as in rule-based approaches, but 
on the construction and manipulation of mental mod- 
els. A mental model represents a possible “state of affairs” 
described in the premises of an inference problem. It 
only represents what is true according to the premises 
but not what is false. The common assumption of most 
mental-model accounts (Johnson-Laird, 1983; Johnson- 
Laird & Byrne, 1991) is that reasoning is a cognitive pro- 
cess in which spatially organized or iconic models of the 
premises are constructed and then alternative models 
are sequentially generated and inspected. A conclusion 
is true if it holds in all models that agree with the prem- 
ises (Johnson-Laird, 1999, 2001; Johnson-Laird & Byrne, 
1991; Johnson-Laird & Khemlani, 2013). This theory 
is supported by many experimental results and is also 
implemented in several computer programs (Bara, Buc- 
ciarelli, & Lombardo, 2001; Khemlani & Johnson-Laird, 
2013; Krumnack, Bucher, Nejasmic, Nebel, & Knauff, 
2011; Ragni & Knauff, 2013). Chapter 2.3 by Johnson- 
Laird, chapter 3.3 by Khemlani, chapter 6.3 by Byrne 
and Espino, chapter 13.2 by Ragni, and chapter 13.3 by 
Knauff (all in this handbook) are related to this theory. 

Rule-based theories and the model theory also explain 
why human reasoning is sometimes error-prone—the 
second question that a descriptive theory of reasoning 
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should account for. In one respect, both theories agree: 
both assume that, in theory, humans have the compe- 
tence to think and act rationally according to certain 
normative standards, but that, in practice, this compe- 
tence is limited by many internal and external condi- 
tions. Apart from this commonality, however, there are 
large differences between the different theories. 

According to rule-based theories, errors can creep in 
due to the variety of processes that are necessary for rea- 
soning. Where exactly they may creep in is determined 
by the core assumptions of the theory. Before a mental 
representation of the premises can be stored in work- 
ing memory, the premises must first be encoded by pro- 
cesses of understanding. Here, interpretation errors may 
occur because the use of logical expressions in natural 
language differs from that in logic. Once the premises 
are available in working memory, abstract reasoning 
schemata must be applied to the premises to derive valid 
inferences. Here, the wrong rules might be selected or the 
coordination of different rules might fail. And invalid 
conclusions may be produced by still other processing 
errors (Braine, 1990; Braine & O’Brien, 1998; Rips, 1994). 

In the model theory, reasoning errors are mainly 
caused by the limited capacities of working memory: 
reasoning becomes more difficult when multiple mod- 
els need to be generated from the premises, and it takes 
additional time to discover inconsistencies between 
tokens in a model. Errors can occur when potential 
alternative models are not generated or inconsistencies 
are overlooked. A crucial assumption is that inferences 
derived from initially formed explicit models are simpler 
than inferences that can be performed only by elabo- 
rating other implicit models. The distinction between 
implicit and explicit models is described in chapter 2.3 
by Johnson-Laird (this handbook). In the past years, the 
preferred models theory has explained why people usually 
prefer to construct just a single easy, simple model of the 
given information, why they ignore alternative interpre- 
tations that are also logically valid, and why this leads to 
inferences that deviate from the norms of classical logic 
(Knauff, 2013; Ragni & Knauff, 2013). 

We should also mention that other theories claim 
that humans are fundamentally irrational. Some of 
these theories are about theoretical rationality, but they 
are particularly prominent in research on practical ratio- 
nality, notably in areas overlapping with behavioral eco- 
nomics. The most prominent accounts are the different 
versions of dual-process theories, according to which 
people reason by means of two different systems or pro- 
cesses. Kahneman (2011) argues that one system, Sys- 
tem 1, processes the incoming information fast, largely 
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unconsciously, and relatively effortless. It relies on heu- 
ristics that often lead to good solutions but sometimes 
also to violations of rational norms. The other system, 
System 2, processes the incoming information slowly, 
consciously, and relatively effortfully. This more “ratio- 
nal” system might rely on logical rules, mental models, 
probabilities, and so on. Because people tend to use Sys- 
tem 1 more often, they are considered largely irrational. 
Although the account is intuitively appealing—which 
might be one reason for its success inside and outside 
psychology and even in the public—it has many empiri- 
cal and theoretical problems (Gigerenzer, 2010; Osman, 
2004). Today, most reasoning researchers prefer to speak 
of two “processes” rather than “systems” and also admit 
that the simplified functional distinction in earlier 
theories cannot be upheld (J. St. B. T. Evans, 2018). In 
chapter 2.5 of this handbook, Klauer gives an up-to-date 
review of dual-process theories of human reasoning. 

How do reasoning processes interact with prior beliefs, 
background knowledge, or attitudes? We have already 
said a bit on this topic in subsection 2.2 of this intro- 
ductory chapter, but we want to emphasize again that 
no descriptive reasoning theory denies that in daily life, 
all of these strongly interact. Proponents of the Bayes- 
ian approach often allege that other theories do not pay 
attention to knowledge and prior beliefs. Yet, this is not 
really true. In fact, a more adequate description is that 
present reasoning theories differ in the role they assign 
to prior beliefs. Broadly construed, one position is that 
reasoning should be studied and described in its pure 
form. In such accounts, prior knowledge is a moderat- 
ing factor that can influence reasoning by supporting or 
hindering it. This is where the term “bias” comes from. 
The other position is that people do not do much rea- 
soning anyway but mostly use their beliefs to deal with 
inference problems. “Reasoning” is basically the retrieval 
of beliefs from memory and some less relevant cognitive 
procedures to process this knowledge. Most serious theo- 
ries lie in between the two extremes, and there are several 
approaches that can deal with the effects of prior beliefs 
and the resulting nonmonotonicity of human reasoning 
within logical theoretical frameworks (see chapter 5.4 by 
Gazzo Castaneda & Knauff, this handbook). 

Let us also briefly look at the descriptive side of practi- 
cal rationality. Again, we can be shorter here, although 
the issue is broader, because we now deal not only with 
epistemic matters but also with how people actually 
judge and decide based on their desires, attitudes, values, 
and so forth. That is, we discuss the topics of judgment 
and decision making under the heading of practical 
rationality, even though philosophers would carefully 
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distinguish between decisions and judgments. Here, we 
just stay away from vocabulary and briefly look at the 
psychology of judgment and decision making. Nor do 
we look at what some psychologists and neuroscientists 
call “decisions” too, for example, to lift one’s arm or not, 
or to choose between two equally meaningless stimuli 
on the left and right side of the visual field (Heekeren, 
Marrett, & Ungerleider, 2008). Such “perceptual deci- 
sions” are largely arational and thus not considered here. 

In principle, we could ask the same questions for 
practical rationality as we did for theoretical rationality: 
how can we explain people’s competence, their perfor- 
mance, and the interaction of the underlying cogni- 
tive processes? The situation is a bit more complicated, 
however. One reason is that prior knowledge and beliefs 
are now complemented, and maybe even overruled, by 
values, attitudes, and so on. Another reason is that the 
discussions that we already mentioned on the normative 
side carry over to descriptive theories of human judg- 
ment and decision making as well. In fact, there is hardly 
any consensus concerning the best norms by which the 
judgments and decisions of people should be assessed. 
Accordingly, it is quite hard to say what a “good” judg- 
ment or decision is and what we should call an “error.” 
So, many deep questions touching upon moral, ethical, 
cultural, economic, political, and ideological issues are 
involved in such appraisals. Hence, we better stay away 
from such evaluations in the next few paragraphs. 

Broadly construed, a judgment occurs when some- 
body assigns a value to an object on a particular dimen- 
sion. The objects of judgment can be things, situations, 
actions, people, abstract stimuli, and so on. In a judg- 
ment, the attitudes, values, preferences, etc. of a person 
become visible. So, philosophers would rather call it a 
“value judgment.” 

The most important theoretical framework for human 
judgment is the Jens model, which was developed around 
the 1950s by Egon Brunswik (1956) and promoted by 
Kenneth R. Hammond (1996) in research on social judg- 
ments. The basic idea of this quite metaphorical model is 
that people cannot perceive the environment directly and 
objectively but instead use cues to make inferences, judg- 
ments, and choices. Some of these cues are more obvious 
in the environment while others must be inferred from 
further cues. People must select the cues they consider 
relevant, assign relative weights to them, and finally use 
these cues as aids in the judgment process. Since the real 
properties of the object are not directly accessible to the 
perception of the judging person, the cues’ usefulness 
depends on how well they represent the real properties 
of the judged object. This is why Brunswik is considered 
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one inventor of the concept of ecological validity, which 
indicates how accurately the cues represent the actual 
properties of the object within the given environment. 
The higher the correlation is, the more the cue helps to 
come to an appropriate judgment. 

The lens model is also popular in descriptive theories 
of human decision making. When individuals decide, 
they must choose between at least two different alterna- 
tives, for example, going to the anti-glyphosate demon- 
stration or having a relaxed Sunday on the sofa. Whereas 
judgments stand for evaluations and preferences, deci- 
sions indicate an intention to pursue a particular course 
of action (Hardman, 2009, p. 3). The common feature is 
that both processes are based on cues that reflect more 
or less well the actual characteristics of the judged object 
or decision alternatives. 

Descriptive theories of decision making seek to explain 
how people identify different choice options, how they 
gather and weigh information and cues, how they evalu- 
ate the feasibility and desirability of the alternatives, etc., 
in order to make choices that lead to the best outcome, 
including all costs and benefits. It is obvious that this 
is the place where all the normative questions reappear, 
which we ignore in this subsection. What we can say, 
however, is that an important turnaround happened 
when Kahneman and Tversky (1979) developed their 
prospect theory (see chapter 8.3 by Glockner, this hand- 
book), in which they argued that the formal structure of 
probability and utility functions is empirically incorrect 
and hence proposed more adequate alternatives. Since 
then, several descriptive theories have been developed 
that also account for decisions that they may or may not 
want to call “errors” because they deviate from the classi- 
cal economic norms. Good examples are the approaches 
of bounded rationality (see chapter 8.5 by Hertwig & 
Kozyreva, this handbook), which criticize the standard 
decision theory as computationally overdemanding. And 
since classical economists are prone to load utility func- 
tions with egoistic or self-concerned connotations, it is 
easy to denounce the thus-laden theory as highly ideo- 
logical, which in a way questions its normative status 
(see subsection 5.1 of this introductory chapter). 


4.3 On the Relation between Descriptive 

and Normative Theories of Rationality 

How are normative and descriptive theories of ratio- 
nality related to each other? For Piaget, the answer was 
clear: typically developed adolescents and ultimately 
adults should attain the ability to reason according to 
the rules of formal logic. In this view, human reason- 
ing research actually is, and should be, driven by the 
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norms of logic and by a comparison of mental reason- 
ing with these logical norms (Inhelder & Piaget, 1958). 
Certainly, this is a strong idealization, which not many 
people would support today. We cannot go into all the 
details of this complicated methodological issue.”” What 
we can do, however, is to briefly present our opinion on 
the matter, which, we feel, does not accord with any of 
the opinions we find in the literature. 

First, the normative and the descriptive perspectives 
are both legitimate and important. This goes without say- 
ing for the descriptive perspective. And it holds for the 
normative perspective precisely in the genuine, internal 
sense explained at the beginning of this section. We are 
bound to act, and thus we are bound to take a norma- 
tive stance—even the decision to let things go is a nor- 
mative stance. So, the normative perspective is absolutely 
unavoidable. And it extends to rationality, if rationality 
is normative. This raises the issue of how the normative 
and the descriptive perspective on rationality are related. 

One possibility is that the two perspectives are simply 
independent. Apparently, they are at least logically inde- 
pendent. Anyone inferring an ought from an is commits 
a naturalistic fallacy—this philosophical lesson from 
Hume and Moore seems to stand.*° And surely, reversely 
inferring an is from an ought is no less of a fallacy. These 
assertions are cleared up from the logical point of view 
in Schurz (1997), although there are quite a few logical 
subtleties. As a consequence, Elqayam and Evans (2011) 
argue that much of cognitive rationality research com- 
mits these fallacies, to its detriment. The authors do not 
deny the legitimacy of the normative point of view, but 
they suggest that empirical research better makes itself 
completely independent from it. 

Another possibility is that “rationality” simply means 
different things in the two perspectives: there may be 
rationality; and rationality. (J. St. B. T. Evans & Over, 
1996). Then, however, our joint handbook would merely 
rest on a big equivocation. Neither possibility gives us 
the positive relation we are looking for. 

A natural way of building a bridge between the nor- 
mative and the descriptive side, particularly in our scien- 
tifically minded times, is to try to naturalize rationality. 
“Naturalized epistemology” is a slogan introduced by 
Quine (1969), and if the quote from Quine (1986) in 
subsection 3.3 of this introductory chapter were true, 
epistemic rationality would indeed reduce to “technol- 
ogy,” to a branch of empirical science. In the same vein, 
Schurz and Hertwig (2019) explain rationality in cogni- 
tion to be instrumental for the predefined aim of cogni- 
tive success, which is measured in terms of ecological 
validity and applicability. 
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Cohen (1981), although not avowedly in the natu- 
ralistic camp, in effect argues in a similar way. As men- 
tioned in subsection 4.1 of this chapter, he distinguishes 
a narrow and a wide reflective equilibrium in normative 
theory construction. The narrow equilibrium takes nor- 
mative intuitions or primitive normative assessments 
as fixed reference points and attempts to systematize 
only them. By contrast, the wide equilibrium accounts 
for more comprehensive considerations even of an 
expert nature, e.g., logical theorizing or only theoretically 
explicable principles of minimal change. Then norma- 
tive intuitions may be negotiable. At the end of subsec- 
tion 4.1 of this chapter, we, too, used the metaphor of 
a reflective equilibrium. There, we definitely intended 
it in the wide sense. Cohen, by contrast, ties rational- 
ity theory firmly to what he calls the narrow equilib- 
rium. Thereby, however, normative theory construction 
amounts to empirical theory construction over the fixed 
normative reference points: it determines our cognitive 
competence. The cognitive performance may deviate—a 
little. But the competence can never turn out to be irra- 
tional. This is Cohen’s way of empirically controlling 
normativity. Again, we do not find genuine normativity 
in Cohen’s picture. 

In our view, these strategies of naturalistic reduction 
do not properly respect the autonomy of the normative 
discourse, which is so amply displayed in this handbook. 
Let us explain this with respect to the three naturalistic 
strategies just mentioned. 

First, if rationality judgments were just a matter of a 
priori normative intuition, as suggested by Schurz and 
Hertwig (2019), their criticism of the normative per- 
spective would be justified. But they are not; they are a 
matter of normative argument. They are also right when 
they complain that the metaphor of reflective equilib- 
rium entails circular justification. But that’s perhaps the 
nature of justification, and we simply need further crite- 
ria to distinguish good from bad circles. In their alterna- 
tive account of rationality as instrumental for cognitive 
success, they simply take the aim of cognitive success 
for granted, and thus rationality research reduces to the 
exploration of this instrumental relation. We, by con- 
trast, want to maintain the autonomy of normative dis- 
course by interpreting their account as a normative one. 
It may well be normatively convincing to strive for cog- 
nitive success in their precise sense, weighing ecological 
validity and applicability, and to include this success in 
our normative reflective equilibrium. But this requires 
further normative argument. 

Second, the quote from Quine (1986) in subsection 3.3 
of this introductory chapter underrates this autonomy in 
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a similar way. Beliefs aim at truth. Granted, but perhaps 
this slogan does not assign to epistemology the task of 
providing a “technology” for reaching a predetermined 
aim. Perhaps the aim itself is explicable only within 
the reflective equilibrium for epistemic rationality (see 
Spohn, 2016). 

Third, Cohen, too, seems to underrate the autonomy 
when he firmly ties rationality to normative intuitions. 
But these intuitions, even though we have no better 
starting point, are often dim or confused, and the task 
is not just to quasi-empirically systematize but to nor- 
matively develop and explicate them. Thereby, they 
become an indispensable ingredient of the wide norma- 
tive reflective equilibrium. 

Our rejection of naturalistic reduction and our empha- 
sis on the autonomy of the normative discourse can be 
seen to reflect Kant’s doctrine of the autonomy of pure 
practical reason (Kant, 1788/1908): it has the positive 
freedom of self-legislation, and it has the capacity and the 
mission to fill that freedom. However, this emphasis at the 
same time deepens the gulf separating it from the empiri- 
cal realm. As stated, logic cannot bridge the gap between 
the normative and the descriptive. What else can? 

The crucial point, we think, is that there are plenty 
of defeasible (not deductively valid) inference relations 
between the normative and descriptive perspectives on 
rationality. In practice, we find these inferences every- 
where, but we do not find this point clearly addressed 
in the methodological literature. In moral philosophy, 
another chapter of genuinely normative theorizing, 
people talk of prima facie reasons. For instance, if some- 
thing is (descriptively) pleasing, then, prima facie, it is 
(normatively) good. This assertion is not analytically 
true—it is an open question whether something pleasing 
really is good. But the assertion is plausible—and accept- 
able when there are no opposing reasons. 

The general reason for these defeasible inference rela- 
tions is that humans are receptive to norms. We tend 
to follow the norms, not only the norms somehow 
externally given but also the genuine norms internally 
endorsed. This receptivity is sometimes strong, some- 
times weak. There are uncountable confounding factors. 
Still, the receptivity exists. The point is often expressed’! 
by stating that the normative theory simultaneously 
serves as an empirically idealized theory. This entails, 
quite generally, the following defeasible connection: 
whenever something that is under human control ought 
to be the case, there is some plausibility that it actually 
is the case. And reversely, whenever something under 
human control is the case, then there is some plausibil- 
ity that it ought to be the case. This may sound far too 
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strong, and you will be able to immediately cite hun- 
dreds of counterexamples. Granted, but the abstract 
point could only be put so starkly. Of course, it must be 
applied with all due caution and reservation. But this 
does not make the point go away. 

In rationality research, this point is ubiquitous. Phi- 
losophers explicate reasoning in the hope that people 
then obey their explications. And psychologists refer to 
normative standards in order to specify precisely this 
defeasible relation. Now, whenever there seems to be a 
discrepancy between norm and empirical fact, an excep- 
tion to our defeasible rules, there are several ways to go. 
There is, first, the standard way of sticking to the norm 
and finding additional explanations why observed facts 
deviate from the norm. The Wason selection task, for 
example, leaves the norms of deductive logic unshaken 
and postulates certain biases disturbing the performance. 
In this case, the criticism concerns the people: they are 
somehow irrational. There is, second, the possibility 
of modifying the normative reference point in order 
to reduce or possibly eliminate the discrepancy. This 
was the Bayesian strategy: perhaps people’s inferences 
should not be tested against logic but against probability 
theory. Then the criticism turns against the scientist for 
choosing an inappropriate normative reference point. 

So far, the discrepancies had no consequences for the 
normative discussion. Neither logic nor probability the- 
ory are demoted in their normative status by the Wason 
task. It is just a matter of finding out empirically which 
normative ideal we should take people to adhere to and 
how much deviation from the normative ideal we should 
admit. The default assumption is perhaps: the less devia- 
tion, the better. But this is not so clear—the psychology 
of reasoning might still have to find its own descriptive 
reflective equilibrium. 

Then, however, there is the third case, where the 
discrepancy is taken as a normative criticism. Take the 
Popper-Kuhn controversy (subsection 2.1 of this intro- 
ductory chapter): Kuhn described the behavior of sci- 
entists. Popper took the first way and criticized this 
behavior as irrational. But this response seemed implau- 
sible, since the scientists did not accept the charge of 
irrationality. So, it rather seems that something is wrong 
with Popper’s normative theory. Or take the Ellsberg par- 
adox (subsection 4.1 of this chapter): people turn out to 
be stubborn. Even when we explain to them why their 
preferences are allegedly irrational, they stick to them. 
So, are they stubbornly irrational? Perhaps something is 
wrong with standard decision theory if it does not dis- 
tinguish between known risk and a numerically identi- 
cal subjective probability. This is an empirical input, 
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which needs to be respected in the normative discussion. 
Here we have indeed a defeasible inference from is to 
ought. 

It is unnecessary to give further instances of the vari- 
ous ways of this defeasible inference. If we are aware of 
its possibility, we easily see it everywhere. Defeasibility 
is not a one-way bridge. Defeasible inference carries us 
both ways, from the normative to the descriptive, and 
inversely. Empirical theorizing has to find its own reflec- 
tive equilibrium (most descriptions of scientific meth- 
odology are much more detailed, though). Normative 
theorizing has to find its own reflective equilibrium as 
well, as sketched above and exemplified in the hand- 
book. However, the two equilibria are correlated. One 
cannot optimize the one in disregard, or at the cost, of 
the other.” This is very vague but already gives an idea 
of our main point. It is methodologically important to 
observe the logic of descriptive-normative reasoning 
and to keep this double equilibrium in mind. 

Elqayam and Evans (2011) fear to thereby get entan- 
gled in the arbitration problem, the problem of deciding 
between competing normative accounts: “Psychologists 
can ...and do get involved in arguments about which 
norm is right—perhaps an odd activity for empirical 
scientists” (p. 239). The arbitration problem is indeed 
severe and will perhaps never be conclusively decided. 
Still, we think this activity is not odd at all but unavoid- 
able.** It is an activity, or discourse, that philosophers 
and cognitive psychologists should jointly engage in. It 
is necessary in order to make progress in our understand- 
ing of human rationality. In fact, this handbook is one 
large plea for this joint enterprise. 


5. Individual and Social Rationality 


There is another basic classification of theories of ratio- 
nality, namely, into those about individual and those 
about social or collective rationality. This classification 
cuts across the distinctions we have already introduced. 
We find a lot of empirical research on both individual and 
social rationality, as well as on many normative issues. 
And just like individual rationality, social rationality has 
an epistemic as well as a practical dimension. 

At first glance, the distinction is clear: individual ratio- 
nality is about the epistemic or practical reasoning pro- 
cesses of a single person and their outcomes, while social 
rationality is about group processes, their results, and their 
rational organization, concerning the purely epistemic 
formation of a joint opinion as well as the agreement on 
a joint strategy or the settlement of practical conflicts. 
We have seen many issues and instances of individual 


Psychological and Philosophical Frameworks of Rationality 


rationality in the previous sections. In fact, those sections 
were severely biased toward individual rationality, while 
neglecting the social dimension of rationality. 

There are reasons for this bias. The main reason is that 
philosophy and psychology both have an individualistic 
bias. Our Humean heritage gives a nice little example of 
this bias. On the one hand, one should think that com- 
munication provides the foundation for all social episte- 
mology. On the other hand, Hume (1748/1975b, section 
X, “Of Miracles”) classified testimony (through listening 
and reading) as a mode of perception among others, to be 
compared with perceptions in other modes, and the pro- 
cessing of any kind of perception is a matter of individual 
rationality. If so, social epistemology seems reduced to 
individual epistemology. Not that we would maintain this 
nowadays, but the bias persists. Another reason is that the 
notion of social rationality is much less clear than that of 
individual rationality, as we will indicate below. So, the 
field of social rationality is still more tentative. 

For these reasons, this handbook has an individu- 
alistic bias as well. However, the social dimension of 
rationality is so important that we also wanted it to be 
represented in this handbook, even if only in a more 
exemplary way (see sections 9, 10, 12, and 14 in this 
handbook). In the following, we want to discuss at least 
briefly some central issues of social rationality. 


5.1 Individual Rationality in a Social Context 

Game theory is the classical paradigm of rational choice 
theory in a social context.** It has developed into the 
foundational economic theory and pervades economic 
theorizing. It is clearly about practical rationality, and it 
is clearly about how to behave rationally in social con- 
texts, that is, when other agents are involved. However, 
is it also about collective rationality? 

There is no clear answer. In order to understand this, 
we must look at the basic distinction between coop- 
erative and noncooperative game theory. Cooperative 
game theory is not necessarily about cooperation. But 
it doesn’t turn cooperation into a problem. It assumes 
that communication among all participants or players is 
free and available and that agreements can be reached, 
which may or may not be enforceable. This may, but 
need not, result in cooperation. Perhaps you only com- 
promise with your opponents, or you find partners in 
order to defeat your opponents, and so on. Large parts 
of cooperative game theory are indeed about suitable 
coalition-forming. Thus, it certainly addresses important 
forms of collective rationality. 

Similarly, noncooperative game theory is not neces- 
sarily about noncooperative agents. Clearly, players are 
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allowed to cooperate also in noncooperative games. 
However, cooperation requires action, e.g., communi- 
cating, bargaining, entering into agreements, etc., and 
from the point of view of the noncooperative theory, 
each kind of action must be explicitly represented as a 
move in the game. This is why game theorists consider 
the noncooperative part as basic and the cooperative 
one as derived. And it is for this reason that only nonco- 
operative game theory is represented in this handbook. 

Now, it is important to see that noncooperative game 
theory is not about collective rationality. It is rather 
about individual rationality in a social context.*> How 
can you pursue your interests in an environment of 
other agents who also pursue their interests? This is not 
simply a matter of instrumental rationality, of manipu- 
lating our fellows like instruments. It means recognizing 
that our fellows try to reach their aims as well. It is, one 
might say in Kantian terms, a matter of treating our fel- 
lows as ends and not as means. 

In this perspective, the notion of a Nash equilibrium 
takes center stage. The actions or strategies of the players 
are in equilibrium if, given the choices of the other(s), 
no one can improve by changing her choice. Only then 
does everybody individually optimize. Also, only such 
equilibria can be publicly known and shared—no pub- 
lic advice can deviate from such an equilibrium without 
giving to at least one player a reason to reject the advice. 

It may be asked whether such individual rationality 
in a social setting is really something special. Classical 
game theory assumes so up to the present day and con- 
siders standard decision theory as the limiting case of 
“one-person games” or “games against nature.” Bayes- 
ian game theory, as developed by Harsanyi (1967, 1968a, 
1968b), and its successor, epistemic game theory (Bern- 
heim, 1984; Pearce, 1984; Spohn, 1982), try to reverse 
the direction of derivation and to conceive of equilib- 
rium behavior as ordinary expected utility maximiza- 
tion under certain special conditions (see chapter 9.2 by 
Perea, this handbook, for how far this strategy carries). 
Quite a different interpretation of noncooperative game 
theory is found in evolutionary game theory (see chap- 
ter 9.3 by Alexander, this handbook), which is particu- 
larly germane for evolutionary explanations of various 
cognitive features (see chapter 1.3 by Schurz and chap- 
ter 10.6 by Cosmides & Tooby, both in this handbook). 

The fact that individual rationality in a social setting 
does not per se amount to social or collective rational- 
ity becomes obvious in the prisoners’ dilemma, which is 
perhaps the most famous paradigm of game theory:*° 
two prisoners in two separate cells may or may not con- 
fess an alleged common crime. In other words, they can 
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either defect (from his partner in crime, D = confess) or 
cooperate (with his partner in crime, C = not confess). If 
a prisoner is the only one to confess, he is set free as a 
key witness (this has the highest utility, 3—the exact fig- 
ures are not relevant), while the other one is imprisoned 
for a long time (this has the lowest utility, 0). If both 
confess, they get imprisoned for a slightly shorter time: 
at least they have admitted their guilt (utility 1). And if 
both don’t confess, they can be convicted only of a lesser 
crime, entailing a small punishment (utility 2). The story 
has thousands of variations and occurs to us every day. 
The only important matter is the structure of the situa- 
tion, the distribution of utilities, as given, for example, 
in table 1. There, the utilities of the row chooser are 
given in the lower left corner of each field and those 
of the column chooser in the upper right corner. Obvi- 
ously, the only Nash equilibrium is (D, D). However, the 
justification of D is even stronger: each player fares bet- 
ter by defecting, not only given that the other defects, 
but whatever the other does: either he gets 3 instead of 2 
or 1 instead of 0. In other words, cooperating is strictly 
dominated by defecting. Hence, there is almost unani- 
mous agreement that defecting is the only rational thing 
to do, at least if the game is played only once; iterated 
playing is a more complicated issue. From the point of 
view of individual rationality, this seems undisputable. 
From the point of view of collective rationality, however, 
this is silly. Both would fare better by cooperating: they 
would get 2 instead of the 1 from the equilibrium. 

The prisoners’ dilemma is often characterized as a 
conflict between individual and collective rational- 
ity. But what is collectively rational? In the prisoners’ 
dilemma, it seems obvious that joint defection is collec- 
tively irrational because it is strictly Pareto-dominated, 
as economists say: both could fare better by jointly 
cooperating. However, not being Pareto-dominated (i.e., 
being “efficient”) provides at best a necessary condition 
for collective rationality, indeed a very weak one. Note 
that the C/D combinations are also efficient. In fact, it 
is quite unclear what collective rationality might mean 
beyond the criterion of efficiency, and in some contexts, 
the latter is even doubtful as a necessary condition (e.g., 


Table 1 
Prisoner’s dilemma 


Player B 
C D 
Player A C 2,2 0, 3 
D 3,0 pea | 
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in the liberal paradox invented by Sen, 1970, chapter 6). 
However, this only shows that social rationality is a 
much more difficult and obscure notion than individual 
rationality. We will return to this later. 

What is the normative status of game theory? It is quite 
unclear. The vacillating interpretation of rationality as 
individual or social adds to the uncertainty. However, the 
uncertainty persists even if we focus on individual ratio- 
nality. Noncooperative game theory clearly has strong 
normative foundations, but there are also many situations, 
like the iterated prisoners’ dilemma and the ultimatum 
game (Gtith, Schmittberger, & Schwarze, 1982), where the 
recommendations of game theory are ignored by agents 
and indeed doubtful even from a normative point of view. 
Therefore, many economists have become quite guarded 
about game theory as a normative theory and prefer to 
characterize it as an idealized model that can claim valid- 
ity only under restricted conditions. How far this validity 
extends, though, has become quite contentious in turn. 

Most of the descriptive work in the field uses experi- 
mental paradigms inspired by noncooperative game 
theory. Dozens of experiments use different versions of 
the prisoners’ dilemma, the ultimatum game, and many 
other games, to study the roots of rational cooperation 
and conflicts between individuals and social groups 
(Axelrod, 1997). 

We should emphasize, however, that such experi- 
ments were criticized for many reasons (Bowles, 1998; 
Gomberg, 1989; Ostrom, 1998, 2010). The main criti- 
cism is that they derive from the tradition of classical 
economic thinking and the self-adjusting mechanisms 
of free markets. Indeed, it may be argued that an indoc- 
trination with rational choice theory biases people 
toward acting more selfishly than they would other- 
wise. Therefore, one may suspect that the theory and its 
related paradigms create the type of selfish people they 
axiomatically assume (Frey & Meier, 2002). The main 
cause for this criticism is that economic theory tends to 
load the notion of utility with egoism and to assume 
that people decide like selfish maximizers, promoting 
only their individual benefit. However, many empirical 
results in the social and behavioral sciences, particularly 
in behavioral economics, show that this draws a too one- 
sided picture of human rationality, downplaying the fact 
that collaboration lies at the heart of the human species 
and our society (Tomasello, 2009a, 2009b). 


5.2 Social Rationality 

Let us now turn to proper social rationality. This is 
about the attempts of social groups to form joint beliefs 
and joint preferences and to perform joint actions in a 
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rational way. When the group cooperates to arrive at 
rationally justified beliefs, conclusions, or explanations, 
we speak of epistemic social rationality. When the group 
cooperates to form joint decisions, we talk about practi- 
cal social rationality. A group cooperates if each group 
member coordinates its activities with that of other 
group members, and all group members work toward a 
goal that benefits the entire group (Witte & Davis, 1996). 

There are many situations in our society where 
groups of people work together to achieve shared epis- 
temic goals. For instance, a group of jurors wants to 
reach the verdict “guilty” if the defendant is guilty and 
“not guilty” if he is innocent. The members of an ethics 
committee, in psychology or medicine, reason together 
to anticipate possible risks resulting from experimen- 
tal treatments or medical therapies. Scientists working 
together on a joint publication draw inferences from 
empirical data and want to come to justified conclu- 
sions and explanations. Politicians and parliaments seek 
to find the right means to reduce climate change or to 
fight pandemics. It is obvious that we urgently need a 
better understanding of this form of social rationality, 
since it lies at the heart of democratic societies, which 
assign central legislative, executive, and judicial deci- 
sions to groups of people, ranging from boards, cabinets, 
commissions, and parliaments to the entire electorate. 

The easiest way of responding to normative questions 
here might be to use the same standards as for the epis- 
temic rationality of individuals. The problem with this 
transfer, however, is that individual rationality is usually 
defined with respect to the formation, combination, and 
revision of individuals’ beliefs, but it is unclear whether 
it makes sense to attribute beliefs to collectives (e.g., 
List & Pettit, 2011; Theiner, Allen, & Goldstone, 2010). 
Some accounts have been suggested, but none of them 
are as prevalent as the logical and Bayesian frameworks 
of individual epistemic rationality (see also chapter 10.1 
by Dietrich & Spiekermann, this handbook). 

So, let us look at the descriptive side of epistemic 
social reasoning, where interacting groups evaluate the 
“correctness,” “truth,” and “believability” of epistemic 
inferences and beliefs. Only a small number of studies 
investigate this topic. One stream of research started 
from work by Patrick R. Laughlin and coworkers, who 
showed that the mere announcement of a purely “intel- 
lectual task” with an “objectively correct solution” 
triggers other social combination processes than the 
announcement that different opinions should be merged 
to come to a joint decision (Laughlin & Adamopoulos, 
1980; Laughlin & Ellis, 1986). The authors use the con- 
cept of “demonstrability” to explain which beliefs are 
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shared by a cooperating group: a “demonstrably” correct 
solution is the one for which the group members can 
demonstrate that it is correct. They also found a correla- 
tion between demonstrability and the number of group 
members required for a collective solution (Laughlin & 
Ellis, 1986). Similarly, a study by Moshman and Geil 
(1998) on deductive reasoning in groups indicates that 
groups are better at avoiding typical reasoning biases. 
They asked 143 college undergraduates—32 individuals 
and 20 groups of 5 or 6 interacting peers—to solve the 
Wason selection task and found a clear group advantage. 
After intensive deliberation, groups turned out to be less 
prone to the confirmation bias than individuals. There 
are a few more studies on the epistemic rationality of 
groups, but overall, this field has started to flourish only 
recently (e.g., Claidiére, Trouche, & Mercier, 2017; Jern, 
Chang, & Kemp, 2014). 

A second branch of research is in the tradition of 
Gricean and neo-Gricean theories of communication 
(Grice, 1975, 1989; Levinson, 2000), the “relevance the- 
ory” (Sperber et al., 2010; Sperber & Wilson, 1995), or the 
“argumentative theory of reasoning” (Mercier & Sperber, 
2017). All these approaches to communicative rational- 
ity could be said to view human conversation as a form 
of social epistemic rationality. Some of these approaches 
are discussed in chapter 5.5 by Hahn and Collins and 
chapter 5.6 by Woods (both in this handbook). 

Research on practical social rationality is concerned 
with many topics that are not important when groups 
perform epistemic tasks. This research has in fact a 
much longer tradition; there is a huge amount of litera- 
ture from philosophy, psychology, economics, political 
science, etc. Nevertheless, the area is still quite diffuse, 
and the empirical and the normative side seem less well 
integrated than in the case of individual rationality. 
One reason is perhaps that in the field of practical social 
rationality, normative benchmarks stand out much less 
clearly than in that of individual rationality. For this rea- 
son, we have treated the topic in this handbook rather in 
an illustrative than in a fully representative way. 

On the normative side, about the earliest attempts to 
get a theoretical grip on the topic come from democratic 
voting and social choice theory, which are not repre- 
sented in the handbook (see List, 2013). They stand in 
the utilitarian tradition and try to determine (procedures 
for finding) the common good or a social preference on 
the basis of individual preferences. Group decision mak- 
ing has the similar problem of integrating diverging aims 
into one common aim and of finding a joint strategy for 
proceeding. Such group decision processes often presup- 
pose the sharing of information and the integration of 
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diverging opinions into a common one. This is a central 
issue in social epistemology; see chapter 10.1 by Dietrich 
and Spiekermann (this handbook). 

Clearly, these fields are full of normative issues. 
One of the earliest and most baffling ones was Arrow’s 
impossibility theorem, showing that there is no way of 
aggregating arbitrary individual preferences into a social 
preference, provided this aggregation is to satisfy certain 
apparently very plausible conditions, one of which is 
non-dictatorship (i.e., that the social preference is not 
identical with the preference of a given individual or 
“dictator”; List, 2013, section 3). It is much less clear, 
though, whether such normative issues are issues of 
rationality. In the case of group opinion and group deci- 
sion making, the answer seems to be positive: they seem 
quite analogous to individual decision making. How- 
ever, already the case of preference aggregation seems 
different. Is the non-dictatorship condition (if it can 
be interpreted as the absence of a dictator in the ordi- 
nary sense) a demand of rationality? Hardly. It rather 
appears to be a moral requirement. In other words, in 
these social matters, normative issues tend to take on a 
moral character. And the extent to which morality is a 
question of rationality is very much open. If we could 
reduce morality to rationality, moral maxims would be 
just as well justified as postulates of rationality. Presum- 
ably, though, morality should keep its autonomy. This 
unclear situation is another reason why this handbook 
does not fully engage in normative social issues. At least 
section 12 of this handbook is devoted to the relation 
between rationality and morality. 

Cognitive psychologists have also investigated prac- 
tical rationality and decision making in groups. Most 
of this research is motivated by the hope that social 
rationality is in principle superior to that of individu- 
als. However, the empirical findings suggest otherwise: 
while several experimental studies showed that some- 
times groups do indeed perform better than individuals, 
other studies did not find any group advantage, or even 
reported that individuals outperform groups. 

The superiority of groups can have several reasons. 
One prominent finding is that groups often outperform 
individuals because groups tend to weight the input from 
more competent members more strongly. This often 
leads to a “truth wins” principle in a social combination 
process in which the existence of a single knowledgeable 
group member is necessary and sufficient for a correct 
group response (Laughlin & Ellis, 1986). However, Schul- 
tze, Mojzisch, and Schulz-Hardt (2012) have provided 
an alternative explanation, namely, that group mem- 
bers actually become more accurate individually during 
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group interaction. Many other comparisons between 
groups and individuals are reported and often suggest dif- 
ferent explanations for group advantages (Liang, More- 
land, & Argote, 1995). The “wisdom of crowds” literature 
(Surowiecki, 2004) shows that collectives can decide 
rationally, or optimally, even when the individuals in the 
collective are irrational (Lyon & Pacuit, 2013). Yet, this 
only works when the decisions of the group members are 
independent from each other. Good overviews about the 
advantages of groups in judgment and decision making 
can be found in Esser (1998) and Witte and Davis (1996). 

Inferiority of groups can also have several reasons. For 
instance, classical studies have demonstrated that confor- 
mity can lead to suboptimal group performance (Asch, 
1951, 1956). Janis (1972, 1982) showed that members 
of groups often avoid raising controversial issues due 
to the desire to preserve harmony. This “groupthink” 
can lead to irrational decisions in groups of potentially 
rational agents—and has been considered the main rea- 
son for the fact that in 1986, critical information was 
neglected when planning the launch of the space shuttle 
Challenger, which ultimately cost seven astronauts their 
lives (Moorhead et al., 1991). Research in the hidden- 
profile paradigm—in which some information is shared 
among group members, whereas other pieces of infor- 
mation are unshared—demonstrates the potential disad- 
vantages of group collaboration (Brodbeck, Kerschreiter, 
Mojzisch, & Schulz-Hardt, 2007; Jonsson, Hahn, & 
Olsson, 2015; Kerr, MacCoun, & Kramer, 1996; Kerr & 
Tindale, 2004; Mojzisch & Schulz-Hardt, 2006; Stasser 
& Titus, 2003). Research in social-choice and game the- 
ory demonstrates that groups of fully rational agents 
can end up making decisions that are collectively irra- 
tional (Bacharach, 2006; McAdams, 2009). Today, we 
know many more biases that appear in groups and pre- 
vent them from making good decisions. For instance, 
we could hardly call it rational if the order in which 
group members speak influences the joint decision. In 
group meetings, however, those who speak first often 
have a higher impact on the outcome of the group deci- 
sion. This often leads to suboptimal output of the group 
(Bazerman, 2012; Hartmann & Rafiee Rad, 2020) and 
may polarize the group members (Zhu, 2013). Again, 
this can make it hard for groups to reason and decide 
optimally or rationally. 

In sum, the study of social rationality, both from 
the normative and from the descriptive perspective, is 
a highly complex issue that we are just beginning to 
understand. It is also related to many other research 
fields in social psychology, economics, political science, 
and even anthropology and legal theory. This does not 
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make the topic simpler. It is obvious, however, that such 
questions are immensely important for our society, and 
it is good that recently more psychologists and philoso- 
phers have become interested in these topics. 


6. Rationality in a Process-Oriented 
and an Output-Oriented Perspective 


We now want to return to the important distinction 
between output-oriented and process-oriented approaches 
to human rationality. At first glance, the two perspectives 
seem quite closely attached to the two disciplines at the 
center of this handbook: output-oriented approaches 
dominate in philosophy, and process-oriented approaches 
are crucial for the cognitive psychology of human ratio- 
nality. Although these connections to the two disciplines 
are not entirely wrong and can largely be seen in the 
historical developments of both disciplines, we will later 
show that the distinction also cuts across the disciplines: 
some psychologists are also satisfied with output-oriented 
approaches, and process-oriented approaches are, to some 
extent, also pertinent in philosophical approaches to 
rationality. 

Our distinction between theories that focus either on 
the output or on the process of cognition is of course 
not new. Marr’s (1982) well-known distinction between 
the computational and the algorithmic level of description 
is certainly the most influential version of this distinc- 
tion, at least in cognitive science. In subsection 2.2 of 
this introductory chapter, we already explained why we 
nevertheless decided to use another terminology here: 
Marr’s terminology often leads to misunderstandings, 
it is too closely attached to the computer metaphor of 
human cognition, and our terminology leaves more 
space for the vital normative—descriptive distinction. 

Of course, Marr’s approach had precursors. Already 
in the early days of modern logic, there was, on the 
one hand, the so-called syntactic approach, accord- 
ing to which logic just consists in the manipulation of 
symbols in conformity to certain rules, for example, in 
writing strings of symbols (i.e., formulae) that have the 
shape of axioms and then transforming them in ways 
defined by inference rules; no understanding of the for- 
mulae is involved. On the other hand, there was the so- 
called semantic approach, according to which formulae 
receive truth conditions (i.e., meanings), and inference 
is defined by necessary truth preservation, which leaves 
open how the inference is syntactically, that is, algorith- 
mically, realized. The two approaches indeed diverge, as 
became clear through the epoch-making incompleteness 
results of Kurt Godel (1931): logical or mathematical 
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provability does not exhaust logical or mathematical 
truth! This in turn led to investigation of what is com- 
putable, i.e., algorithmically accessible, in the first place. 
Three very different theories—the theories of Turing 
machines, recursive functions, and A-definability—were 
proved to be equivalent, which corroborated the Church- 
Turing thesis that each of these theories indeed captures 
our intuitive notion of computability (see Kleene, 1967, 
chapter V; Kripke, 2013). This was the birth of all our 
modern conceptions of algorithms. 

Another, related terminological distinction is the 
separation of the symbol level and the knowledge level in 
the classical work by Newell and Simon (Newell, 1982; 
Newell & Simon, 1972). On Marr’s computational level, 
we ask, “What is the goal of the computation?” We ask 
“why” a cognitive process is performed by the cogni- 
tive agent. This is similar to Newell’s knowledge level, 
which is an abstract level of description for knowledge, 
intentions, and reasons. On Mart’s algorithmic level, 
the question is: “How are the goals and strategies at the 
computational level algorithmically realized?” which 
is similar to Newell’s symbol level. On this level, we ask 
“how” the cognitive system works: which properties 
the representations have, how the relationship between 
input and output is mediated, and which transforma- 
tions are made to get from the former to the latter. 

The distinction is also reflected in more recent philo- 
sophical writings, where we find the distinction between 
reason-based and mechanistic models of a given phe- 
nomenon. Bechtel and Abrahamsen (2005) state that 
process-oriented explanations in the life sciences often 
consist of models of the mechanisms taken to be respon- 
sible for a given phenomenon. This differs from the 
output-oriented, nomological explanations commonly 
presented in philosophy. Bechtel and Abrahamsen iden- 
tify several differences between these approaches. First, 
whereas in philosophy, the focus on language is quite 
typical, scientists who develop mechanistic explanation 
are not limited to linguistic representations. Second, the 
fact that mechanisms involve assumptions about com- 
ponent parts and operations provides direction to both 
the discovery and testing of mechanistic explanations 
(Bechtel & Abrahamsen, 2005, p. 421). Finally, mecha- 
nistic approaches are developed for specific cognitive 
phenomena and are initially not phrased in terms of 
universally quantified statements. Later generalization 
then involves the investigation of both the similarities 
of a new phenomenon with those already studied and 
the differences between them. In the past decades, Martr’s 
approach and the related suggestions to distinguish 
between different levels of explanation have received 
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many variations, extensions, and criticisms (Peebles & 
Cooper, 2015). A good overview is given in the collec- 
tion of articles in Colombo and Knauff (2020), which 
brings together about a dozen authors from philosophy, 
psychology, biology, AI, robotics, anthropology, and 
other fields. 

We have already mentioned many examples of out- 
put- and process-oriented approaches to rationality. In 
principle, when cognitive psychologists explore rational 
reasoning, be it of a logical, conditional, causal, probabi- 
listic nature or whatever, they aim at understanding the 
underlying cognitive representations and processes. The 
difficulty of this attempt, however, arises from the fact 
that human reasoning belongs to the area of complex 
cognition, a particularly challenging subfield of cognitive 
psychology. One crucial feature of complex cognition is 
that it is embedded in a multitude of cognitive processes 
interacting with one another and with other, noncogni- 
tive processes. For instance, reasoning involves a large 
amount of strongly interlinked processes of language pro- 
cessing, perception, working memory, long-term memory, 
etc. (Knauff & Wolf, 2010). We return to some of these 
subsystems in the next section on the preconditions of 
rationality. In the past, a small number of models tried to 
account for this plurality of processes, but this was often 
at the expense of the theoretical rigor of these approaches 
(D6érner, 1999). Most approaches nowadays focus instead 
on subcomponents of reasoning and decision-making 
processes, with the goal of complementing them stepwise 
with additional processes. Most cognitive approaches in 
this handbook belong to this class of theories. Another 
process-oriented approach is the theory of computational 
rationality, which combines models from AI, cognitive 
science, and neuroscience to reconceive processes of rea- 
soning and action under uncertainty through the lens of 
computation (Gershman, Horvitz, & Tenenbaum, 2015). 
We had a chapter on this approach planned, but, unfor- 
tunately, it did not work out. 

We have also mentioned many examples of philoso- 
phers’ attempts to state and justify normative standards 
for these reasoning processes. Typically, those accounts 
took an output-oriented perspective: They were about 
establishing a criterion for logically valid deductive infer- 
ence, about probabilistic norms for degrees of belief, 
about establishing maximizing expected utility as the 
rational way to act, and so on. And they were about 
all the possible alternatives and amendments to these 
standards. Such approaches also make clear that the two 
perspectives apply to practical rationality just as well as 
to theoretical rationality. Of course, there are reasoning 
processes on both sides. However, the process-oriented 
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perspective may be more difficult to implement on the 
practical side. 

We mentioned already in subsection 3.1 of this 
introductory chapter how the distinction between the 
process- and the output-oriented perspective relates to 
the distinction between internally and externally driven 
dynamics of epistemic states: the internal dynamics is 
about how our thinking does or should proceed, and it 
is thus bound to take a process-oriented perspective. By 
contrast, the external dynamics is about how we do or 
should respond to external input; in determining this 
response, we primarily take the output-oriented perspec- 
tive, although we may and should investigate by which 
internal dynamics the response is mediated. 

We would like to point out, though, that there is a 
deeper question underlying our considerations, namely, 
the question of how to conceive of an epistemic state (or 
a mental state in general) at all. This question is not pur- 
sued in the handbook and will only be touched on here. 

Philosophers are used to distinguishing two con- 
ceptions of epistemic (or mental) states: as something 
occurrent or as a disposition. For them, a state is occur- 
rent if it presently “goes through our mind” or “is before 
our eyes,” that is, if we are presently aware of it. Thus, 
philosophers tend to conceive occurrent epistemic states 
as conscious. For instance, when asked for your opin- 
ion about glyphosate, or any other question, you form 
an occurrent thought or belief, which did perhaps not 
occur to you a moment before, and you are aware of 
this belief. We treat an epistemic state as dispositional, by 
contrast, if we take it to consist in a behavioral disposi- 
tion that may manifest in our verbal and other behavior, 
to be elicited by questions or tests, but without further 
input that could change the disposition. The disposi- 
tion lies, so to speak, in the background of the occurrent 
beliefs, and we are typically not aware of it. 

To illustrate this distinction: in subsection 3.1 of this 
introductory chapter, we distinguished between belief 
bases and belief sets. A belief base can be understood in 
the occurrent sense, as just a few beliefs of which we are 
currently thinking and which we may then algorithmi- 
cally evolve. By contrast, a belief set is deductively closed, 
that is, it contains all logical consequences of the belief 
base. Thus, we can never have it wholly present in our 
conscious mind. Belief sets can be understood only in the 
dispositional sense. Similarly, if the Bayesian conceives an 
epistemic state as a probability measure, he can do so only 
in the dispositional sense. At any given moment, at most 
tiny parts of such a measure can be grasped. 

The distinction between occurrent and dispositional 
belief has been developed independently of cognitive 
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psychology.’ It is tempting, however, to identify occur- 
rent and dispositional beliefs with representations in 
working memory and in long-term memory, respectively. 
Occurrent beliefs might be those that are momentarily 
active in working memory, often conceptualized as a 
“global workspace” (Baars, 2002; Dehaene, Changeux, & 
Naccache, 2011). Only the representations of beliefs in 
working memory are consciously accessible and available 
for cognitive processing. Process-oriented approaches 
from cognitive psychology are concerned with the for- 
mat, representation, update, and revision of such tem- 
porarily active, occurrent beliefs in working memory. 
Dispositional beliefs, in contrast, may be seen as contents 
of long-term memory to which we do not have immedi- 
ate access. Here, our epistemic states are represented as a 
kind of accumulation of myriads of occasions in which 
the relevant topics have been presented to our mind. So, 
consistency checks, for instance, can be performed only 
for the occurrent beliefs in working memory but not for 
all beliefs represented in long-term memory, even though 
this might be normatively desirable. 

Some of these ideas about the relationship between 
the philosophical concepts of occurrent and dispositional 
beliefs and the psychological conceptions of memory 
can also be found in Goldman (1978). Here we may leave 
open whether this cognitive account perfectly matches 
the philosophical intentions. In any case, the lesson for 
us is: When our study of rationality pursues an output- 
oriented or a process-oriented perspective and explores 
an internally or an externally driven dynamics, we 
implicitly presuppose this or that conception of epis- 
temic states, and we must be clear beforehand which 
conception we apply. 

Many of our previous thoughts may suggest that phi- 
losophers are only interested in the output-oriented per- 
spective and psychologists only in the process-oriented 
one. Yet, this is not entirely true. Many micro-theories 
from the early stages of psychological reasoning research 
took an output-oriented perspective (for an overview, see 
J. St. B. T. Evans et al., 1993). Even today, most psycho- 
logical studies about choice are still about the resulting 
choices and not about the underlying thought processes. 
Similarly, we have mentioned that the “new paradigm,” 
which is closely associated with Bayesianism, is also just 
output oriented—and our criticism was precisely that it 
does not attempt to move onward to the process-oriented 
level. So, clearly, we also find output-oriented approaches 
in psychology. 

Conversely, philosophers are also interested in the 
process-oriented perspective, and this is where approaches 
from philosophy overlap with research in AI, for which 


33 


the process-oriented perspective on algorithms is even 
the defining characteristic. This overlap is due to the com- 
mon ancestry in logic. Indeed, in logic, a precise defini- 
tion of logical validity (output oriented) was preceded by 
an account of valid proof procedures (process oriented). 
Many surprisingly different proof procedures have been 
invented for logics with all kinds of purposes (see chap- 
ter 3.1 by Steinberger, this handbook). For instance, axi- 
omatic calculi are very convenient for metamathematical 
purposes, while they are hardly manageable for actual 
proofs. No empirical claim is associated with such algo- 
rithms. Their point is, rather, to establish the feasibility 
of the standards assumed, for example, those of logical 
validity in the case of logic. This feasibility is not guaran- 
teed at all, as is displayed by the incompleteness results 
in logic (see Shoenfield, 1967, chapter 6), and recursion 
theory tells us that the computable functions form only a 
small subclass of all possible functions (Shoenfield, 1967, 
chapter 7). 

The interest in algorithms goes far beyond deduc- 
tive logic. AI researchers try to find efficient algorithms, 
and philosophers belabor the wide field of philosophi- 
cal logics, where we find epistemic, conditional, coun- 
terfactual, and nonmonotonic logics, among others (see 
chapter 5.1 by van Ditmarsch, chapter 5.2 by Rott, chap- 
ter 6.1 by Starr, and chapter 7.1 by Pearl, all in this hand- 
book). There is also probability logic (see chapter 4.4 
by Pfeifer, this handbook). And the so-called roll-back 
analysis of decision trees (Raiffa, 1968) is an algorithm 
for finding a strategy with maximal expected utility. It 
works perfectly, but precisely its unrealistic nature has 
motivated the search for more realistic accounts in the 
field of bounded rationality (Rubinstein, 1998). 

This interest in algorithms is not an empirical one, 
and it is not backed up by any empirical research on 
thought processes. Indeed, today few AI researchers 
share the concerns of cognitive science. They are rather 
interested in the space of possibilities for specifying pro- 
cesses and finding algorithms that arrive at the intended 
results and besides have other kinds of desirable prop- 
erties. Similarly, philosophers often discuss normatively 
plausible axioms before having a semantic benchmark, 
and if they possess it, they are interested in its computa- 
tional feasibility—algorithmic inaccessibility might, but 
need not, be regarded as a blemish on the benchmark. 
In any case, we may conclude that the process-oriented 
and the output-oriented perspective are not exclusively 
assignable to psychology or, respectively, philosophy. 

This brings us to the main issue of this section, the 
relation of the output- and process-oriented perspec- 
tives to the normative-descriptive distinction, which 
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is so central to our enterprise. Certainly, one may sus- 
pect strong relationships between the two dichotomies. 
One obvious connection is that the conclusions of our 
reasoning processes should satisfy certain normative 
benchmarks. Such normative evaluations always take 
place within the output-oriented perspective. Since the 
normative evaluations lie mainly in the scope of phi- 
losophy, this would explain why philosophy is mostly 
engaged with this perspective. Conversely, the study of 
our actual reasoning processes can, it seems, only be an 
empirical, that is, descriptive, matter. 

By and large, this is correct. However, we should 
note that the normative perspective also extends to 
the process-oriented level. Of course, we normatively 
evaluate also algorithms in themselves and not only 
the results they deliver. Computer scientists do this all 
the time. However, the relevant normative consider- 
ations in AI differ from those in philosophy. Typically, 
the former are about computational complexity classes, 
that is, about the time and storage space needed by an 
algorithm to solve a problem. For computer scientists, 
complexity considerations are based on the behavior 
of an algorithm in worst-case scenarios (Papadimitriou, 
1994). The nightmare for computer scientists are NP- 
hard problems, which cannot in general be solved effi- 
ciently. Although such results from complexity theory 
take a process-oriented perspective and are obviously 
relevant for software development, it is questionable 
whether computational complexity is indeed relevant 
from a cognitive point of view. One criticism is that it 
might be better to look at the average performance of 
algorithms (Sedgewick & Flajolet, 2013). Another criti- 
cism might be that people usually do not have to deal 
with an infinite number of choice alternatives, which 
typically is the basis of complexity theory. An example 
is the traveling salesman problem (finding the short- 
est possible route that visits a given set of cities), which 
is an NP-hard problem but is manageable when just a 
small number of cities must be visited. Moreover, people 
often do not aim for optimality but rather are satisfied 
with a good solution. Altogether, complexity theory pro- 
vides potentially interesting process-oriented normative 
insights for rationality research, but it is questionable 
how relevant such findings are for descriptive theories of 
rationality. Some psychologists emphatically deny their 
relevance (Gigerenzer et al., 2011), while others consider 
complexity theory highly relevant for the cognitive sci- 
ences (Otworowska, Blokpoel, Sweers, Wareham, & van 
Rooij, 2018). 

A different example of a normative process-oriented 
perspective is given by the attempts in decision theory 
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to consider decision costs. The various ways of reaching 
a decision require efforts of varying costs, which should 
enter into the expected utilities of the available options. 
Yet this idea is apparently threatened by circularity. The 
costs of a decision procedure depend on the decision 
problem at hand, and to determine them, one would 
need to already have completed the decision procedure 
for the given problem. For this reason, the problem 
seems quite intractable (Bossaerts & Murawski, 2017; 
Gottinger, 1982). A radical response to it consists in sat- 
isficing: by fixing in advance an appropriate satisfaction 
level, we avoid the costs of further optimization. 

The literature on bounded rationality is quite ambigu- 
ous with regard to the normative-descriptive distinction. 
Treatments from the economic side can also be read as 
normative recommendations for how to get along with 
all kinds of restricted resources (Rubinstein, 1998), while 
psychologists usually explore how people proceed under 
actual restrictions concerning time, working memory, 
etc., without evaluating the thinking processes them- 
selves (see also chapter 8.5 by Hertwig & Kozyreva, this 
handbook). Such evaluations come up only when we 
ask how to improve our thinking or our rational capaci- 
ties in general (see chapter 15.2 by Stanovich, Toplak, & 
West, this handbook). 

So, what is the connection between normative and 
descriptive theories, on the one hand, and output- and 
process-oriented accounts of rationality, on the other? 
The lesson from this section is that that the two distinc- 
tions are independent and that all four areas generated 
by them are important and indeed pursued in both psy- 
chology and philosophy. The two disciplines have dif- 
ferent preferences; this is nothing to complain about. 
However, it would be particularly helpful if the field 
were to make progress also on the normative side of 
process-oriented theories and explain what good think- 
ing ought to be like, not only regarding the output but 
also regarding the thought processes themselves. Such 
a development would be helpful not only for educa- 
tion and the development of curricula in schools and 
universities: it would indeed help all of us if we better 
understood how we should think in order to develop 
and update rational beliefs and come to good decisions 
and actions. 


7. Preconditions of Rationality 


This section is concerned with a topic that also could 
have stood at the very beginning of this introductory 
chapter: What are the preconditions of human ratio- 
nality that distinguish us from other animals? What 
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enables people to think and act (more or less) rationally? 
Of course, the answer to these questions is: our larger 
brains! Certainly, the increase in brain size is the most 
striking feature of human evolution. Probably no other 
organ in mammalian evolution has evolved as quickly 
as the human brain: the brain volume of the prehomi- 
nid Australopithecus africanus was about 400 to 500 cm’, 
which is about the brain size of today’s chimpanzees; 
the brain size of the first tool-making Homo habilis was 
between 500 and 750 cm?; and that of Homo erectus, 
who lived about 1.5 to 0.3 million years ago, reached 
a volume between 880 and 1,100 cm*. The now liv- 
ing humans have a very wide range of brain volumes. 
According to Beals et al. (1984), the average worldwide 
brain volume is 1,350 cm%, although the existing dif- 
ferences are substantial—the brain volume of a healthy 
adult can lie between approximately 900 and 2,000 cm’. 

Obviously, brain volume alone is not decisive for cog- 
nitive ability. In absolute terms, the sperm whale has the 
largest brain, and even the brains of horses and elephants 
are larger than those of humans. In relative terms—when 
we relate brain size to body mass—the shrew mouse has 
the largest brain, although it does not have the cognitive 
abilities of humans. Among humans, males have about 
10% larger brains than females—but on average, there 
are no differences in cognitive abilities between men and 
women as assessed by traditional intelligence tests. Now- 
adays, most researchers say that about 10% of cognitive 
ability can be explained by brain size (Pietschnig, Penke, 
Wicherts, Zeiler, & Voracek, 2015). 

In fact, the connection between brain size and cog- 
nitive abilities is much more complex. Some studies 
show that the correlation is higher when specific areas 
of the brain are considered. For instance, cognitive abili- 
ties are more strongly correlated with the size and thick- 
ness of the prefrontal cortical areas (Menary et al., 2013), 
they are correlated with the connectivity of neurons in 
the brain (Cole, Yarkoni, Repovs, Anticevic, & Braver, 
2012; Geng et al., 2018), and there is also evidence that 
the brains of people with lower cognitive abilities con- 
sume more energy in particular brain areas than the 
brains of people with higher cognitive abilities (Neu- 
bauer & Fink, 2009). However, even hard-core neurosci- 
entists acknowledge that we need much more research 
to better understand the connection between cognitive 
abilities and the anatomy, structure, and functioning of 
the brain (Luders, Narr, Thompson, & Toga, 2009). This 
may be one reason why, in trying to understand ratio- 
nality and other cognitive capacities of humans, many 
cognitive psychologists for a long time refrained from 
studying the brain. 
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Another reason for the long-lasting skepticism of 
cognitive psychologists toward brain research is even 
more fundamental. It lies in the assumption of multiple 
realizability, which was the core argument for the rise 
of functionalism in cognitive psychology and philosophy 
of mind. As a rule of thumb, we can say that the more 
complex the cognitive ability was that the research- 
ers were interested in, the less they believed that these 
abilities could be understood by studying how the brain 
works. Since rationality ranges very high on this scale 
of cognitive complexity, this position was particularly 
prominent in rationality research. Right around the 
millennium, this changed, and today many researchers 
use functional brain imaging and other neuroscientific 
methods to study the cognitive and cortical foundations 
of human rationality. In the following sections, we dis- 
cuss which of these systems appear to be so fundamental 
that they can actually be regarded as essential precondi- 
tions of rationality. We also argue that the development 
of Homo sapiens into a highly social being was an essen- 
tial precondition for the evolution of rationality. 


7.1 Cortical Preconditions of Rationality 

The first neuroscientific studies on rationality focused 
primarily on epistemic rationality. Meanwhile, however, 
there are also many findings on the neural precondi- 
tions of practical rationality. The early experiments with 
functional brain imaging, which we can assign to the 
field of epistemic rationality, have primarily investigated 
logical thinking. The pioneering work was carried out by 
Vinod Goel (see chapter 1.4 by Goel, this handbook) and 
our own group (e.g., Knauff, Mulack, Kassubek, Salih, & 
Greenlee, 2002). It would not be helpful to present all the 
complex neural activation patterns that have since been 
reported in many neuroscientific articles. These results 
can be found in excellent meta-analyses of the several 
dozen published studies on logical reasoning with con- 
ditionals, relations, and quantifiers, which used quite 
different methods for presenting premises and asking for 
conclusions (Goel, 2007; Monti, Osherson, Martinez, & 
Parsons, 2007). Instead, we believe, it is much more use- 
ful to bring some structure to the field by describing the 
core networks associated with logical thinking. 

We think there are three: the first core network cov- 
ers large areas of the brain that have often been asso- 
ciated with the processing and production of language. 
These include areas in the left frontal and prefrontal 
cortex (PFC), also comprising Broca’s area, a region 
involved in semantic tasks and language production 
(Goel, Gold, Kapur, & Houle, 1998; Prado et al., 2015). 
Although this system is highly sensitive to the format of 
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the presentation and the like (hence the activations vary 
enormously in the studies), the fact that many imaging 
studies found activations in these brain regions is usually 
interpreted as indicating that logical thought processes 
are based on linguistic processes and representations. It 
is obvious that this is consistent with one of the central 
theoretical assumptions of rationality research, to which 
we return later. 

The second core network involved in logical reasoning 
covers areas in the parietal cortex, the inferior temporal 
cortex, and the occipital cortex, which are involved not 
only in visual perception but also in visual mental imagery 
(Fangmeier, Knauff, Ruff, & Sloutsky, 2006; Knauff, 2009; 
Knauff, Fangmeier, Ruff, & Johnson-Laird, 2003; Knauff 
et al., 2002; Ruff, Knauff, Fangmeier, & Spreer, 2003). 
These findings are often interpreted as indicating that in 
logical thinking, not only linguistic processes are involved 
but also processes in which people vividly imagine what 
would be the case if the premises of a given logical prob- 
lem were true. These mental simulations—or models, or 
images—can be of very different precision. Some may be 
almost like representations caused by visual perception, 
while others are less vivid but still much more concrete 
than purely linguistic representations. While linguistic 
representations agree with the intuitions of many rational- 
ity researchers, visual representations fit well with the intu- 
itions of laypeople, who often report experiencing their 
own thinking as “seeing before the inner eye” (Knauff, 
2009, 2013). We will also return to this topic in a moment. 

The third core network includes areas of the brain 
that are associated with executive control processes 
and conflict resolution. These cover areas in the right 
lateral/dorsolateral PFC, detecting conflicts between the 
logical structure of a task and its content. These areas 
are activated, for example, when a conclusion is logi- 
cally valid but implausible, because it does not agree 
with our beliefs. Typically, this is associated with con- 
tent effects at the cognitive level and the so-called belief 
bias. Since these experiments were carried out mainly by 
Goel and his collaborators, they take up a lot of space in 
his chapter (chapter 1.4 in this handbook). In addition, 
these findings are related to other results showing that 
in conflicts between form and content, the normatively 
correct answer is available in the brain—even if it cannot 
subsequently prevail against the intuitively more plau- 
sible response. De Neys interprets this as indicating that 
people have logical intuitions that are deeply rooted in 
our brains (De Neys, 2012). 

A methodological problem with imaging techniques is 
that they only establish correlations (but no causal rela- 
tions) between cortical and cognitive processes. There is a 


Markus Knauff and Wolfgang Spohn 


certain irony in the fact that, despite these relatively weak 
statistical accounts, the results of brain-scanning experi- 
ments are so overrated by many people, scientists and lay- 
people alike (McCabe & Castel, 2008; Munro & Munro, 
2014). One way to more directly study the causal con- 
nections between cortical and cognitive functions is to 
examine patients suffering from damages of certain brain 
regions due to strokes, tumors, accidents, etc. In fact, a 
variety of studies have yielded interesting results that, for- 
tunately, match well with the results from imaging stud- 
ies. For instance, a strong argument for the necessity 
of certain brain regions for a specific cognitive capac- 
ity is given when a specific kind of brain damage leads 
to impairment in tasks of type A but does not in tasks 
of type B, while another kind of brain damage leads to 
impairments in B but not in A. Such double dissociations 
have also shown that damage to certain brain regions 
leads to impairment in linguistic or imagery-based rea- 
soning, while other regions are necessary for abstract or 
concrete reasoning processes that may or may not agree 
with our beliefs (Goel, Shuren, Sheesley, & Grafman, 
2004). Brief overviews of patient studies on rational 
reasoning and its impairments can be found in Knauff 
(2009) and Reverberi, Shallice, D’Agostini, Skrap, and 
Bonatti (2009). 

Patient studies, however, also have problems because 
they are often based on a small number of participants, 
sometimes just on a single case. In addition, the brain 
damage is often not clearly delimited or associated with 
other damages. Recently, new methods have become 
available in which specific brain regions can be selec- 
tively blocked for a short period of time (by applying 
a strong magnetic field). This corresponds to a tempo- 
rary lesion—even if this description is greatly simplified 
and not really correct. In any case, with this method, it 
is possible to investigate which cognitive tasks can and 
cannot be performed by an individual whose informa- 
tion processing in the particular brain region is tempo- 
rarily disrupted. Such studies have also shown that the 
application of the disrupting magnetic field to areas in 
the parietal and visual cortex can affect logical think- 
ing, which indicates that mental imagery indeed plays a 
causal role in logical thinking (Hamburger et al., 2018; 
Ragni, Franzmeier, Maier, & Knauff, 2016). 

Recently, some Bayesians have criticized the fact that 
the background knowledge and prior beliefs of partici- 
pants have not been sufficiently taken into account in 
previous experiments. Moreover, they argued that the 
instructions were always biased toward the standards 
of classical logic and did not allow the participants to 
sufficiently consider their various beliefs when solving a 
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problem (Oaksford, 2015). However, this camp of ratio- 
nality research has not yet carried out its own brain- 
imaging experiments to support these arguments. To 
take up their criticism, Gazzo Castaneda, Sklarek, Dal 
Mas, and Knauff (2021) conducted a combined cogni- 
tive and brain-scanning experiment in which condition- 
als with high and low conditional probabilities as well 
as abstract conditionals were embedded in valid and 
invalid inferences. Some participants had to evaluate 
the probability of the conclusion, while others received 
deductive instructions. Both groups of participants could 
freely choose between a dichotomous and a graded 
response. During brain scanning, the participants hav- 
ing received deductive instructions showed elevated cor- 
tical activity in regions typically associated with conflict 
detection and inhibition of prior knowledge. In con- 
trast, when given probabilistic instructions, additional 
activity was found in areas correlated with the influence 
of prior knowledge. Moreover, the results revealed that 
many participants, even those having received proba- 
bilistic instructions, preferred dichotomous responses. 
Content only affected inferences under probabilistic, 
but not deductive, instructions. These findings suggest 
that people’s consideration of prior knowledge and their 
preference for graded responses are not universal. In 
fact, people can flexibly activate or suppress background 
knowledge when they reason probabilistically or deduc- 
tively. This calls into question how meaningful and 
empirically justified the disputes between the different 
camps in the psychology of epistemic rationality are. 
There are also many neuroscientific studies on practi- 
cal rationality. It has been known for decades that damage 
to the brain’s frontal lobe impairs people’s ability to think 
and make choices and decisions (Stuss & Levine, 2002), 
and these findings were supported by functional brain- 
imaging studies of the intact brain (Koechlin, Ody, & 
Kouneiher, 2003). Many of these studies were concerned 
with the neural correlates of dealing with moral dilem- 
mas, the prisoners’ dilemma, and several other paradigms 
and tasks from social psychology and behavioral econom- 
ics (Sanfey, Loewenstein, McClure, & Cohen, 2006; San- 
fey, Rilling, Aronson, Nystrom, & Cohen, 2003). It is not 
surprising that the pattern of neural activation depends 
strongly on the given paradigms and tasks. Here, again, 
we just want to give a summary of the cortical networks 
whose functioning seems to be a precondition of decision 
making and practical rationality (although neuroscien- 
tists probably wouldn’t use the latter term). 
Neuroscientific studies have usually distinguished 
between two components of decision-making processes: 
the first core network is related to cognitive control 
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functions, including task switching, response inhibition, 
error detection, response conflict, and working memory 
(Miyake et al., 2000). Several studies have found corre- 
lated activity in the dorsolateral PFC and the anterior 
cingulate cortex, as well as other parts of the PFC (Bot- 
vinick, Nystrom, Fissell, Carter, & Cohen, 1999; MacDon- 
ald, Cohen, Stenger, & Carter, 2000; Stuss, 2011). The 
second core network is connected to assigning values to 
choices. These functions have been mainly associated 
with ventral and medial parts of the PFC (Glascher et al., 
2012; Grabenhorst & Rolls, 2011). 

What is interesting about these studies are not the 
exact cortical localizations but rather the finding that 
the cortical networks for “cognitive control” and “value- 
based processes” seem to be quite distinct, even though 
they interact in many ways. This is interesting, because 
it has implications for our understanding of practical 
rationality: the involved processes are not a muddle; 
rather, they can be clearly separated and have differen- 
tial effects on the outcome of the decision process. This 
finding has also been supported by brain-imaging stud- 
ies with patients who had lesions to one or the other 
network (Glascher et al., 2012). That these damages in 
fact resulted in an impairment of either the cognitive 
or the value-based processes supports the assumption 
that these networks are not only correlated but both 
necessary for rational decision making and _ practical 
rationality. 


7.2 Cognitive Preconditions of Rationality 

Having a brain is obviously a necessary precondition 
for rationality. But which cognitive preconditions must 
a “system” meet to enable rationality? Certainly, the 
system must be intelligent, at least to a certain degree. 
However, recent research has demonstrated that the 
concept of intelligence should not be identified with 
the concept of rationality, for many reasons. In particu- 
lar, “intelligence” is an individual-difference concept for 
distinguishing between cognitive abilities of individuals, 
whereas “rationality” is typically not used in this way. 
There is the essential distinction between epistemic and 
practical rationality but no corresponding distinction in 
most intelligence tests. And the concept of intelligence 
frequently served as a door-opener for racism, eugenics, 
and the justification of social injustice, whereas rational- 
ity is not associated with such ideological matters. 

In the present context, however, the most strik- 
ing problem with equating intelligence and rationality 
arises from the many experimental findings signifying 
that even very intelligent people are prone to irrational- 
ity. The present handbook is full of such findings. There 
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are even studies in which classical reasoning problems 
were combined with IQ tests and other measures of cog- 
nitive ability, and they often found that performance 
on such tasks correlates relatively weakly with cognitive 
abilities and often only under very specific task instruc- 
tions. For example, people with higher cognitive abilities 
performed better only if there was an advanced warn- 
ing that biased processing must be avoided (Stanovich 
& West, 2008). Based on this evidence, Stanovich and 
coworkers argued that rationality is a very different con- 
cept than intelligence. A certain level of intelligence is 
a necessary but not sufficient precondition of rational- 
ity, which also relies on the ability for reflective thought 
(Stanovich, 2009) to control and monitor one’s own 
thinking and to correct it from faulty beliefs and rea- 
soning biases. Chapter 15.2 by Stanovich, Toplak, and 
West (this handbook) further explains why rationality is 
substantially different from intelligence. 

Psychologists have extensively studied control and 
monitoring processes under the label metacognition 
(Koriat, 1993, 2018). This research largely relies on the 
investigation of metamemory, which has long recog- 
nized the difference between the processes responsible 
for retrieving information from memory and the pro- 
cesses responsible for monitoring that information 
(Dunlosky & Bjork, 2008; Koriat, 1993). According to 
Koriat, monitoring refers to the “subjective assessment of 
one’s own cognitive processes and knowledge” (Koriat, 
Ma’ayan, & Nussinson, 2006, p. 38). This assessment can 
rely on implicit cues, for example, the ease with which 
a memory comes to mind (Koriat & Ma’ayan, 2005), or 
it can be based on explicit cues, for example, on the per- 
son’s assessment of her own abilities (Prowse Turner & 
Thompson, 2009). For example, a basic metacognitive 
control function is confidence judgment (Koriat, 2008): 
following an initial response, a second response about the 
correctness of the previous response is generated (Flem- 
ing, Dolan, & Frith, 2012). This is often accompanied 
by a metacognitive experience, called the feeling of right- 
ness (FOR), which can signal when additional cognitive 
work is needed (Thompson & Johnson, 2014; Thomp- 
son, Prowse Turner, & Pennycook, 2011; Thompson, 
Therriault, & Newman, 2016). Another metacognitive 
experience is fluency, the ease with which information is 
processed (Unkelbach & Greifeneder, 2013). 

While most of the past metacognitive research was 
concerned with the regulation and monitoring of retrieval 
processes from memory, there has recently been a growing 
interest in the metacognitive processes that accompany 
more complex cognitive tasks, such as problem solving 
and rational reasoning (Ackerman & Thompson, 2017). 
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For instance, Thompson and coworkers have shown that 
metacognitive control processes support individuals in 
solving conditional reasoning problems (Thompson 
et al., 2011). The difference between classical reasoning 
experiments and these studies was that participants were 
asked to generate an initial response, then a FOR rat- 
ing, and then a final conclusion. Thompson et al. (2011) 
found that individuals use FOR as a metacognitive con- 
trol function, which can signal when additional analy- 
sis and cognitive effort is needed, which finally leads to 
better reasoning performance in individuals. More on the 
essential connection between rationality and metacogni- 
tion can be found in chapter 8.6 by Thompson, Elqayam, 
and Ackerman (this handbook). 

Psychological research on metacognition mainly pro- 
ceeds from what we have called the process-oriented 
perspective. However, the issues may also be tackled 
from an output-oriented perspective. This has been 
done by philosophers, although they do not use the 
label “metacognition” but rather speak about “higher- 
order attitudes,” which are concerned with beliefs about 
one’s own beliefs or also about one’s own desires, or 
with desires concerning one’s own desires, etc. Higher- 
order attitudes are hence concerned with all kinds of 
self-referential propositional attitudes. The origin of this 
topic lay in the fact that constructions like “It is neces- 
sary that p,” “It is permitted that p,” “a wants that p,” “a 
thinks that p,” etc. can be iterated, just as we can form 
arbitrarily long conjunctions. For instance, I can say, “I 
think that I think” or “T believe that I want,” etc. So, the 
question arose what the logic of these constructions is, a 
question that is extensively treated in modal and inten- 
sional logic (Montague, 1974; Zalta, 1988). It became 
clear, however, that this question is not just a matter of 
logic, of the linguistic rules governing these construc- 
tions, but also a matter of rationality requirements. For 
instance, it is disputed whether we are always aware of 
our beliefs, i.e., whether when I believe that p, I also 
believe that I believe that p. And conversely, the question 
is whether a second-order belief is rationally infallible, 
i.e., when I believe that I believe that p, then I do in fact 
believe that p (Kemmerling, 2017, chapters 13-20). The 
topic is not restricted to epistemic matters. For instance, 
Frankfurt (1971) has famously proposed that free will 
is essentially connected to our ability to form effective 
second-order desires and volitions. 

Higher-order attitudes have also a dynamic dimen- 
sion. So-called auto-epistemology deals with the connec- 
tions between one’s present first-order beliefs and one’s 
(present) second-order beliefs about one’s future (or 
past) beliefs. For instance, if you believe that tomorrow 
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you will believe that it will rain the day after tomorrow, 
shouldn’t you already now believe that it will rain the 
day after tomorrow? One is inclined to say, “Yes, you 
should” (Binkley, 1968). The issue has interesting conse- 
quences, particularly in probabilistic terms (van Fraas- 
sen, 1984). Finally, interpersonal versions of the topic 
loom large in everyday life: we permanently think about 
the attitudes and beliefs of our fellows, about what they 
think about us, and how common knowledge emerges 
in social or public processes. This is particularly rele- 
vant in social discourse and communication (see chap- 
ter 10.3 by Meggle, this handbook). It is also related 
to the false-belief task, to which we return in the next 
section. Although the task is mainly used to study the 
development of “theory of mind” in children, it has also 
been used to investigate the development of rational- 
ity in children, for instance, when they learn to reason 
counterfactually (Rafetseder, Schwitalla, & Perner, 2013). 

Although the false-belief task is a good example of 
the overlap between philosophy and psychology in 
the study of epistemic rationality, there are still many 
ideas in philosophy that have not yet been taken up by 
psychologists. For instance, the philosophical work on 
one’s present beliefs and one’s beliefs about one’s future 
beliefs would also be very interesting from a cognitive 
point of view. Overall, research on metacognition and 
second-order attitudes and beliefs would have many 
implications, for example, concerning the rationality of 
nonhuman animals: other animals seem to have little or 
no reflective metacognitive capacities. So, does rational- 
ity require such capacities? This raises not only empirical 
but also conceptual questions. 

Let us turn to the relation between rationality and 
memory, one of the grand concepts and research topics 
of cognitive psychology. We can hardly doubt that ratio- 
nality would be impossible without an ability to learn, 
store, represent, and remember knowledge, beliefs, atti- 
tudes, values, and all kinds of mental states. An interest- 
ing observation, though, is that theories of rationality 
are often not connected with accounts of the structure, 
functions, and constraints of human memory. This 
applies particularly to normative and output-oriented 
theories of rationality, notably in philosophy, but not 
only there. These theories presuppose a highly idealized 
model of the agents’ memory. A good example for such 
an idealization is the rational requirement of the consis- 
tency of one’s belief system. Thus, the evaluation of one 
belief requires the evaluation of others to maintain con- 
sistency. While this is a normative demand, the belief 
systems of actual human beings are often not adjusted 
in that way. This becomes apparent when we consider 
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that knowledge and beliefs must be maintained over 
certain periods of time, must be recalled when required, 
and must be consciously manipulated to draw inferences 
and to make decisions. Beliefs can also change over time, 
and, importantly, they can be forgotten or may some- 
times not be retrievable from memory. 

All these processes take place in two subsystems of 
human memory: working memory and long-term mem- 
ory. Working memory is conceived as a memory system 
that allows us to maintain information for certain rela- 
tively short periods. Even though older theories of short- 
term memory specified particular time frames in seconds, 
the really critical issue is how much information needs 
to be maintained and whether new information is com- 
ing in while you are still occupied with maintaining the 
previous information. In theory, information in work- 
ing memory can be rehearsed for a very long period if 
no new information is coming in. The limiting factor 
is thus not time but capacity. An early quantification 
of the capacity limit was suggested by Miller (1956), 
who claimed that the information-processing capac- 
ity of working memory is around seven elements. Later 
research, however, showed that this number depends on 
many other factors; for instance, working memory span 
is lower for long than for short words (Baddeley, 1986). 
The “elements” can be words or visuospatial information 
or any other kind of meaningful verbal or perceptual 
units. They can come bottom-up, from the perception 
of external stimuli, or top-down, from thought pro- 
cesses. Forgetting in working memory can have different 
causes. One cause is that elements in working memory 
decay over time, unless decay is prevented by rehearsal. 
Another cause can be that new elements replace older 
ones, and different elements interfere with each other 
or might compete during the retrieval process. Since all 
these can lead to forgetting, elements in working mem- 
ory can have two fates. If they are not rehearsed, they are 
irretrievably lost. Or they can, via consolidation, become 
elements of long-term memory. 

Long-term memory, in contrast, is the storage system 
for all we have learned through our lifetime. The ele- 
ments in long-term memory might not currently be 
used but are needed to enable rational thinking and 
actions. Elements in working memory can become ele- 
ments of long-term memory when we make meaning- 
ful associations to what is already stored in long-term 
memory. One subsystem of long-term memory— 
called semantic memory by psychologists since Tulving 
(1972)—is a repository for the long-lasting storage of 
world knowledge, ideas, concepts, beliefs, attitudes, 
and everything that we have accumulated throughout 
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our lives that helps us to make sense of the world. For 
example, the knowledge that glyphosate is a pesticide 
is stored in semantic memory and also the belief that 
it may kill bees. All such memory traces are strongly 
interconnected in a network-like structure, which fol- 
lows several complex organizational principles (for an 
overview, see J. R. Anderson, 2000). Another subsystem 
of long-term memory—which psychologists call epi- 
sodic memory—is related to biographical information 
and events, which are represented including the tem- 
poral and spatial context. Surely, the memory traces of 
such personal experiences are of great importance for 
the formation and revision of beliefs, but we cannot 
discuss this topic here. 

The structure and functioning of working and long- 
term memory are often the limiting factors of human 
rationality. They can explain the large gap between 
how people actually reason and how they should reason 
according to certain normative theories. For example, 
when a person is confronted with the set of premises 
of a reasoning problem, the amount of information can 
simply exceed the capacity of working memory. Hence, 
the individual might forget some of the relevant propo- 
sitions and thus come to incorrect conclusions. Or the 
information from an earlier premise might interfere 
with new information, and so forth. There are many 
other reasons for human irrationality that are related to 
the structure and limitations of human memory. People 
might also use strategies to deal with these limitations, 
for instance, by using heuristics, which we have already 
mentioned several times. Mental model theory, too, is 
inspired by the limitations of working memory, since 
deviations from the norms of rational reasoning may 
occur because the number of mental models to be con- 
sidered exceeds the capacity of working memory (Klauer, 
1997; Knauff, Strube, Jola, Rauh, & Schlieder, 2004; Sut, 
Oberauer, Wittmann, Wilhelm, & Schulze, 2002; Van- 
dierendonck & De Vooght, 1997). Other rationality the- 
ories are more strongly related to long-term memory, for 
instance, by assuming that we solve problems by retriev- 
ing domain-specific knowledge that we have acquired in 
the past (Cheng & Holyoak, 1985; Oaksford & Chater, 
2012, 2020). 

It is essential to understand that working memory is 
the window to our beliefs, knowledge, attitudes, opin- 
ions, and so on. It brings together the outer and inner 
world. Only what is momentarily active in working 
memory can be consciously experienced, reflected upon, 
and checked for consistency or correspondence to the 
actual world. It is the “global workspace,” corresponding 
to all the momentarily active, subjectively experienced 
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mental states of a person (Baars, 2002; Dehaene et al., 
2011). In section 6, we already argued that occurrent 
beliefs may be seen as elements in working memory, 
whereas dispositional beliefs are (sets of) elements stored 
in long-term memory. This connection between occur- 
rent beliefs and working memory, on the one hand, 
and dispositional beliefs and long-term memory, on the 
other, fits well with the fact that we are aware of what 
is currently in working memory but do not have direct 
conscious access to long-term memory, just as we are 
not permanently aware of the countless beliefs that we 
have accumulated throughout life. We can, of course, 
become aware of a small subset of beliefs when they are 
transferred into working memory in the service of ongo- 
ing tasks. In principle, such a task can also consist in 
evaluating and readjusting beliefs in the way normative 
accounts propose. In working memory, though, only a 
small number of beliefs can be considered at the same 
time, which obviously provides strong constraints on 
consistency checks as suggested in many theories of 
belief systems and belief revision in philosophy and AI 
research (Alchourrén et al., 1985; Gardenfors, 1992). 
This is an idea that deserves more empirical investiga- 
tion. In any case, there can be no doubt that having a 
functioning memory is a central, if not the central, pre- 
condition that a system must meet to enable rationality. 

You may object that Janguage is at least equally impor- 
tant for rationality. This is certainly true. Indeed, many 
philosophers and psychologists share the view that 
language and thought, and thus rationality, are closely 
related. Already Plato lets Theaetetus consent to the 
claim, “Well, then, thought and speech are the same; 
only the former, which is a silent inner conversation of 
the soul with itself, has been given the special name of 
thought” (Sophist 263e, quoted from Plato, 1921). Simi- 
larly, Aristotle (1984, book III) saw thinking as an ability 
that only those living beings can have who can “con- 
sult with themselves”. This is a widespread position in 
contemporary philosophy and psychology that we can- 
not spell out in detail here. However, the opposite posi- 
tion is that language, thinking, and rationality might 
not be as closely linked as it seems. Pinker (1994), for 
instance, says that the idea that thought and language 
are the same is a “conventional absurdity” (p. 49). With 
this position, of course, we are drifting into a huge field 
of research, where philosophers and psychologists have 
suggested many different theories of human thought 
that are less closely linked to language and more to 
mental models, spatial representations and processes, 
mental simulations, bodily experiences, and so forth (de 
Vega, Glenberg, & Graesser, 2008). 
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Why is the role of language for human rational think- 
ing so controversial? One reason is that many people, 
unnecessarily, associate the topic with their opinions 
about the influence of culture, cultural relativity, and the 
universality of cognitive principles. This leads directly to 
opinions about the relation between nature and nurture 
that have always been more ideological than scientific. 
The other reason, although perhaps not independent of 
the first point, is that empirical evidence is equivocal. 
Indeed, the famous Sapir-Whorf hypothesis, that the 
structure of a language determines its speakers’ way of 
thinking and conceptualizing the world, has led to an 
enormous amount of empirical research. It will not come 
as a surprise that many linguists interpret the results 
as support for linguistic relativism, whereas cognitive 
psychologists tend to think more in terms of universal 
principles that all people have in common, independent 
from the language they may use. The universalistic posi- 
tion can be found in Li and Gleitman (2002), and a rela- 
tivistic reply can be found in Levinson, Kita, Haun, and 
Rasch (2002). 

So, is language a precondition of rationality? Cur- 
rently, the best answer is: yes and no. In the previous 
section, we reported findings from cognitive neurosci- 
ence indicating that one cortical system involved in 
rational thinking comprises several language-related 
brain areas. And there are tons of findings from cogni- 
tive psychology that point in the same direction. On the 
other hand, this does not mean that language is nec- 
essary for all kinds of rational thinking. Can’t children 
think rationally before they learn to speak? Don’t non- 
human animals also exhibit forms of rationality? Maybe 
the equation “thinking = speaking” is too strong, but it 
also would be mistaken to deny that language plays an 
important role in human rationality. 

Visual imagination is another capacity of the human 
mind that many cognitive scientists consider an impor- 
tant precondition of human rationality. And if we accept 
this, it would again challenge the position that thinking 
is determined by language. Sometimes, as we just said, 
people indeed experience their thinking as inner speech, 
but they also very often report seeing “mental pictures” 
during thinking, “seeing before their inner eye,” for 
instance, what would be the case if the premises of a 
reasoning problem were true. They often say that they 
can scan such visual mental images to find new infor- 
mation that is not explicitly given. This idea is as old as 
the idea of thinking in language. Aristotle, for instance, 
used the term phantasma for ideas closely akin to percep- 
tual experiences. Today, most psychologists accept that 
visual mental images are a kind of mental representation 
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that closely resembles perceptual experience. Empirical 
evidence shows that people can scan visual images to 
obtain information that has never been intentionally 
stored, and it shows that the experience of visual mental 
imagery is accompanied by activity in the primary and 
secondary visual cortex, which supports the idea of a 
strong overlap between visual imagination and visual 
perception (Kosslyn, 1994, 2005). 

Today, most cognitive scientists accept that visual 
mental images are a special kind of mental represen- 
tation in human memory. However, as with language, 
there is still disagreement about the actual importance of 
visual imagery for human rationality. On the normative 
side, skepticism prevails. Among philosophers and logi- 
cians, the idea is prominent that rational thoughts must 
be based on some kind of formal language. Although it 
is generally accepted that pictures and diagrams are suit- 
able for explaining logical relationships, they are rarely 
accepted as an independent method of proof. This way 
of thinking has a very long tradition, and there are rela- 
tively few exceptions to this view (see chapter 13.1 by 
Jamnik, this handbook). 

On the descriptive side, the evidence is inconclusive. 
Most cognitive experiments were based on the following 
hypothesis: if visual images are important for rational 
thinking, then tasks with an easily imaginable content 
should be easier to solve than tasks with a content that 
is difficult to imagine. One can already see that this 
question is primarily concerned with content effects, 
which, by definition, should not really play a role in 
logic. However, the empirical findings suggest other- 
wise. Some experiments have shown that logical think- 
ing is indeed easier with easily imaginable materials. 
Other experiments did not find any connection at all, or 
the results even pointed to the opposite direction. These 
contradictory findings have only been resolved in recent 
years, when it was shown that the different results were 
caused by the fact that very different types of imaginal 
thinking were investigated (Knauff, 2013; Knauff et al., 
2003; Knauff & Johnson-Laird, 2002). Summarizing the 
more recent results, it appears that the assumptions on 
the normative side are not entirely wrong. Indeed, some 
studies could show that visual mental images can even 
be a nuisance to logical thinking. This may, however, 
not be true for all kinds of rational thinking. But in gen- 
eral, the results allow two conclusions. On the one hand, 
mental representations, in which people mentally simu- 
late what is or could be the case, may be an important 
precondition for rational thinking. This is, for instance, 
the core assumption of mental model theory. Mental 
simulations help us to think about alternatives to reality, 
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to think counterfactually, to think about the possible 
consequences of our own actions, and also to under- 
stand the mental states of others. On the other hand, 
this does not say anything specific about the represen- 
tational format of such mental simulations or models. It 
is indeed premature to assume that these kinds of repre- 
sentations are equivalent to pictorial mental images (see 
chapter 13.3 by Knauff, this handbook). Rather, several 
experimental findings and computational models show 
that they can be more abstract and only account for 
information that is relevant for the given task (Knauff, 
2013). These findings underscore what cognitive psy- 
chologists repeat—with good reasons—like a mantra: we 
may sometime have conscious access to the outcome of 
our thoughts, but it is almost impossible to experience 
the cognitive processes that have led to this outcome. 
This is why we need both output-oriented and process- 
oriented research to understand the true nature of 
human rationality. Introspection cannot be relied upon; 
it can and does mislead us and makes us believe that our 
thinking works in the way we experience it. 


7.3 Social and Evolutionary Preconditions 

of Rationality 

Our previous thoughts on the preconditions of rational- 
ity have been limited to the individual. For instance, 
visual mental images may be some of the most private 
things we all have, which may be why they are so inter- 
esting and prominent in the psychological literature. In 
subsection 5.2 of this introductory chapter, however, 
we have already stressed the social aspects of rational- 
ity. We would now like to follow up on these thoughts 
and argue that not only the biological and the related 
cognitive development of our brain was important for 
the evolution of rationality. Equally important was the 
development of Homo sapiens into a social being that 
cooperates, thinks about the beliefs and actions of other 
people, and tries to convince them of its own beliefs and 
plans. 

One of the earliest theories in this direction was 
developed by Leda Cosmides and John Tooby (Barkow, 
Cosmides, & Tooby, 1992; Cosmides, 1989). In their evo- 
lutionary theory of rationality, they argue that human 
reasoning abilities have developed because in bartering 
and trading, compliance with contracts must be checked 
and cheaters must be sanctioned. For instance, to hunt 
a mammoth, many men must work together. To accom- 
plish this, other men must guard the village. That is why 
a contract is made: if the hunters share their prey, the 
others take care of their families in their absence. This 
is a rule that must be followed by both sides to ensure 


Markus Knauff and Wolfgang Spohn 


the common good of the entire village. Violations of 
the rule must be punished. According to Cosmides and 
Tooby, that is why, in the course of evolution, a system 
for detecting cheaters has developed in our brain that 
helps us to draw logically valid conclusions. The idea of a 
“cheater-detection module” goes back to the evolution- 
ary biologist Robert Trivers and his theory of reciprocal 
altruism (Trivers, 1971). In the past decades, Cosmides 
and Tooby have developed their account further into 
an adaptationist program, according to which human 
cognitive architecture was shaped by natural selection 
to solve exact and nonintuitive information-processing 
problems. The researchers describe this approach in 
chapter 10.6 (in this handbook). 

Other social theories are based on the close connec- 
tion between thinking, argumentation, and communica- 
tion. For instance, logic helps us in developing a sound 
argumentation; it brings structure to our arguments and 
makes them comprehensible for the interlocutor. We 
generally find illogical and incoherent arguments uncon- 
vincing. Already the father of logic, Aristotle (1985), in 
his Rhetoric, dealt with how to persuade other people of 
one’s opinion with good arguments and correct conclu- 
sions in meetings and discussions (see also chapter 5.6 
by Woods, this handbook). Recently, Hugo Mercier and 
Dan Sperber (2017) have developed an evolutionary the- 
ory of human reasoning according to which the human 
species developed its tremendous reasoning competence 
because logically sound arguments are the best means to 
persuade others. 

Mercier and Sperber (2017) also provide a socio- 
evolutionary account of why reasoning is so often unreli- 
able. Reason, they argue, is not directed toward solitary 
use, toward achieving rational beliefs and decisions by 
our own efforts. Rather, reason helps us to justify our 
beliefs and actions to others, to persuade them through 
argumentation, and to evaluate the justifications and 
arguments that others direct at us. Although we agree 
that social cooperation may be an important evolutionary 
foundation of rationality, there are also many objections. 
The most general reservation applies to all evolutionary 
theories of cognition: they are often “just-so stories,” that 
is, unverifiable narrative explanations for human behav- 
ior (Gould, 1978; R. C. Richardson, 2007). 

Mercier and Sperber also try to explain why reason 
did not evolve in other animals. Their answer is: because 
no other animal has language. This position is related to 
theories that place the close interaction between social 
cognition and language at the center of the evolution of 
cognitive abilities (Heyes, 2012). In essence, these theo- 
ries state that tasks such as hunting a mammoth require 
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a high degree of cooperation, which is only possible in 
combination with language. Of course, other animals, 
such as groups of lions or societies of ants and bees, also 
collaborate, in a more instinctive way, but they do not 
communicate their goals and plans, which is only possi- 
ble by means of language. 

Although we do not have anything like “cognitive 
fossils,” whereas other disciplines can draw on petrified 
remains of plants or animals from earlier geological ages, 
many empirical findings emphasize the importance of 
sociality and cooperation for human theoretical and 
practical rationality. In a nutshell, these studies show 
that many mistakes people make, for example, in con- 
ditional reasoning, no longer occur if the task is placed 
in a social context. A famous study was conducted by 
Cheng and Holyoak (1985), who presented students 
with a social version of the Wason task. Imagine you are 
a customs officer at the airport. You should check the fol- 
lowing rule: “If there is ‘entry’ on one side of the form, 
then ‘cholera’ is in the list of diseases on the other side. 
This is to make sure that entering passengers are pro- 
tected against the disease.” When the participants were 
presented with the corresponding cards for the cases P 
(entry), =P (no entry), Q (cholera on the list), and -Q 
(no cholera on the list), they performed significantly bet- 
ter than in the original Wason task and also better than 
in tasks embedded in a nonsocial context. Other studies 
showed that improved performance in the Wason task 
involving social agreements only occurs when the sub- 
jects put themselves in the role of a “rule supervisor” but 
not when they are only “observers” (Gigerenzer & Hug, 
1992). In subsection 5.2 of this introductory chapter, we 
already reported several studies showing that groups of 
people are better at solving epistemic reasoning tasks 
than the individuals that constitute the group. 

In the same section, we also reported some results on 
the social aspects of practical rationality. Game theory, as 
we already described, is by far the most far-reaching theo- 
retical and empirical framework in this context. A special 
version of it is evolutionary game theory (Weibull, 1997; see 
also chapter 9.3 by Alexander, this handbook), where suc- 
cessful cooperative action has been explored with differ- 
ent kinds of games, e.g., signaling games (D. Lewis, 1969), 
the so-called stag hunt (Skyrms, 2004), and the already 
mentioned prisoners’ dilemma (Axelrod, 1997). These 
studies showed that, depending on the game, successful 
cooperative action can, but need not, emerge in groups 
and need not always be evolutionarily stable. Tit for Tat, 
for instance, can lead to the escalation of conflict. An alter- 
native is the so-called GRIT (“graduated and reciprocated 
initiatives in tension reduction”) strategy, which can lead 
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to stepwise de-escalation. According to this method, it is 
rational for the party willing to mediate to approach the 
opponent by first publicly declaring his willingness to rec- 
oncile and then making as many concessions as possible, 
as long as these do not cause any major damage. If the 
opposing party reciprocates these steps toward reconcilia- 
tion, chances are good that the conflict can be resolved in 
a rational way (Osgood, 1962). 

Even Charles Darwin was aware of the importance of 
cooperation in human evolution. In his groundbreak- 
ing book On the Origin of Species, he wrote that man’s 
low physical strength and speed and his lack of natu- 
ral weapons, etc., are more than compensated by man’s 
social cooperation, which has enabled him to help and 
receive help from others (Darwin, 1859). Apparently, 
this is miles away from the idea of Thomas Hobbes and 
his famous dictum “Homo homini lupus’— 
wolf to man.” It also far away from many, sometimes 
intentional, misinterpretations of Darwin’s theory as a 
scientific justification for egoism and selfishness. Today, 
for many researchers the development of social coop- 
eration has been the main cause for the development 
of the cognitive abilities of humans, which go so far 
beyond those of all other species. The cognitive anthro- 
pologist Michael Tomasello found that other animals 
and even great apes do not, or at least to a much lesser 
extent, have the capacity to understand their conspecif- 
ics as intentional beings (Call & Tomasello, 2008). This 
research is closely related to studies on the false-belief 
task that explores the development of “theory of mind” 
in human infants. For example, in the unexpected- 
transfer task (Dennett, 1978; Perner & Wimmer, 1985), 
a child sees that Mary puts a candy into a red box. Then 
she goes out of the room and the child sees that another 
child moves the candy from the red into a blue box. Now 
the observing child is asked, where will Mary look for 
the Candy? The correct answer, of course, is that Mary 
will look in the red box. Such theory-of-mind tasks have 
received great attention in psychology and philosophy, 
as they require an awareness in the child that another 
individual does not necessarily possess the same beliefs 
or knowledge that they themselves possess. They are also 
important when we want to understand the evolution- 
ary roots of human rationality. From our view, it is hard 
to imagine that rationality could have evolved without 
the ability to infer the mental states of others and one- 
self. Both go hand in hand, and presumably reflections 
on the mental states of others even came first, before 
humans ever started to think about their own mental 
states. In any case, both seem to be essential precondi- 
tions for human rational thinking. More details on the 
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evolutionary foundations of human rationality can be 
found in chapter 1.3 by Schurz (this handbook). 


8. Frontiers of Rationality 


In the previous sections, we have provided a systematic 
overview of the psychological and philosophical frame- 
works for rationality. Most of these frameworks attempt 
to characterize rationality as a general concept that can 
be applied to many domains and fields of human think- 
ing, reasoning, judgment, and decision making. This gen- 
erality is important, because we do not want to have too 
many small, highly specific empirical results about, and 
conceptions of, rationality for every single domain or 
area of discourse. But, one might object, does this gener- 
ality sufficiently recognize that there are many fields and 
applications of accounts of rationality—in everyday life, in 
public discourse, and in the sciences and humanities— 
that might require more specific concepts of rationality? 
In the following, we want to mention at least the more 
important of these domains and discuss whether they 
indeed require special concepts of rationality or can be 
subsumed under the more general theories presented in 
this handbook. That is the first aim of this final section 
of this introductory chapter. And then there are so many 
other topics in philosophy and psychology that overlap 
with rationality. We could not cover all these neighbor- 
ing fields, but we should at least briefly explain how 
these areas are related to the empirical and theoretical 
conceptions of rationality presented in this handbook. 
That is the second aim of this section. 


8.1 Are There Special Forms of Rationality? 

When we ask this question, we first think of counterex- 
amples, for example, logic, which is a general discipline 
par excellence—the canons of logic apply everywhere. 
Similarly, game theory is intended to apply to all agents 
in social situations. It has thus invaded large areas of the 
social sciences (see chapter 10.4 by Raub, this handbook) 
and is not restricted to particular domains, for example, 
to games in the ordinary sense or financial bargaining, 
even though it has to say special things about particular 
situations. And so on. While these theories are very gen- 
eral and largely domain independent, it has been argued 
that there are also domain-specific forms of rationality 
not to be subsumed under the existing general accounts. 
We want to briefly address three of these areas: Do we 
need a particular concept of scientific rationality? Is 
there a special communicative rationality designed for 
social interaction? Do we need a particular account of 
rationality for artificial intelligence? 
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Let us begin with the sciences (both natural and 
otherwise), which consider themselves to be the heart- 
land of rationality. Does this mean that there is a special 
scientific rationality? It is clear where the self-confidence 
of scientists comes from: from the rise of the scientific 
method through figures like Francis Bacon, Galileo 
Galilei, and René Descartes and the subsequent Age of 
Enlightenment and the accompanying successes in the 
explanation, prediction, and manipulation of natural 
phenomena. This was the beginning of the evolution 
of our modern rational practices and our conceptions 
of epistemic rationality. The main goal of science is to 
acquire new knowledge and justified beliefs by means 
of empirical methods and systematic theorizing. This 
epistemic work lies at the heart of science, and scientists 
seem to pursue their epistemic work in a particularly 
rational way. At least, this is the common ideology. 

But scientists also have problems of a practical nature. 
For instance, the choice of research issues is a practical 
decision. Should I try to develop vaccinations against a 
pandemic virus or rather implants for cosmetic surgery? 
Who should benefit from my research, the whole soci- 
ety or a privileged group? These are clearly value-laden 
decisions, for the budgeting institution as well as for the 
individual researcher. The legal claim of the freedom 
of science is a practical decision as well, based on gen- 
eral freedom rights and assumptions about how to best 
motivate and organize research. Many scientists think 
that such practical problems are not scientific problems, 
which could be solved by scientific methods, and hence 
do not lie in their specific competence. Scientific ratio- 
nality does not refer to such problems. 

But then, the so-called value-freedom of science is 
often cited as a hallmark of scientific rationality. This 
attitude was famously promoted by Weber (1917/1973) 
but has also been criticized as being itself a normative 
and value-laden position. Habermas, as one of the main 
critical voices, has emphasized the normative and philo- 
sophical presuppositions of empirical research (Haber- 
mas, 1968). The positivism controversy centered on such 
issues (Adorno et al., 1969). There is also a lot of con- 
temporary discussion about the postulate of the value- 
freedom of science, which is represented in chapter 14.2 
by Bueter (this handbook). 

Thus, it seems that it cannot be denied that scientific 
practice is suffused with norms and values. Can we still 
defend the idea of the value-freedom of science as an 
ingredient of scientific rationality? The only way to do 
so is to limit the idea to the area of epistemic rational- 
ity. Only here can value-freedom be a norm for science, 
and then it means that no value judgment or normative 
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conclusion can be inferred from scientific methods alone. 
Doing so would be to commit the naturalistic fallacy, 
that is, to violate Hume’s principle that no ought can be 
inferred from an is (see subsection 3.3 of this introduc- 
tory chapter and chapter 1.1 by Sturm, this handbook). 
This also implies that scientific knowledge can support 
normative conclusions only in the presence of normative 
premises. These premises, however, must be made explicit 
and transparent to the public. From what we have written 
in subsection 4.3 of this chapter, it is clear that we want 
to modify this requirement. There, we concluded that the 
empirical scientist must also engage in normative theoriz- 
ing, because she must deal with all the defeasible bridges 
between descriptive and normative considerations. 

So, if science has a special claim on rationality, this can 
only refer to the epistemic side. Scientists are supposed to 
be specialists in epistemic rationality. Cognitive psychol- 
ogists, for instance, have highly sophisticated methodol- 
ogies of hypothesis testing, mostly of a statistical and an 
experimental nature. This is what cognitive psychology 
has in common with the other sciences, although each 
field has its own challenges and its own responses. Sci- 
entific methodologies are sophisticated normative can- 
ons of epistemic rationality, and they exceed everyday 
epistemic rationality by far. This is a competence science 
has built up over centuries. Of course, this is not to say 
that scientists always follow these canons. Phenomena 
like “p-hacking” (performing statistical tests until they 
yield a significant result) or publication bias (that mostly 
statistically significant results are published in journals) 
indicate that science also suffers from certain structural 
deficits in epistemic rationality.** 

However, such deficiencies should not distract from 
the special normative competence of the sciences with 
regard to epistemic rationality. Still, this is not yet to say 
that there are special principles of scientific rationality 
(chapter 14.1 by Andersen & Andersen in this handbook 
offers some relevant considerations). Ockham’s razor 
(Sober, 2015), classical statistics, and inference to the best 
explanation (Lipton, 1991) may be claimed to constitute 
specifically scientific methods. The general trend in phi- 
losophy of science, though, is to subsume such scientific 
methods under general principles of epistemic rationality. 
For instance, the Popper-Kuhn controversy mentioned 
above seemed to suggest otherwise. According to Kuhn, 
scientists follow a special practice, which Lakatos (1978) 
and others have attempted to rationalize. This rationaliza- 
tion seemed to be tailored for the sciences. However, it 
later led to investigations into the logic of theory change 
in general, and this in turn resulted in belief revision the- 
ory as displayed in chapter 5.2 by Rott (this handbook) 
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and related accounts. Thus, what seemed special to sci- 
ence has been subsumed under general epistemic rational- 
ity. Another example is provided by the dispute between 
classical and Bayesian statistics, where classical statistics 
defends special principles of statistical inference, while 
Bayesian statistics tries to get along with general principles 
of Bayesian learning. Thus, it is at least open whether the 
sciences can or must appeal to special principles of epis- 
temic rationality. This question is also important when we 
consider the connection between scientific rationality and 
the public understanding of science, which is discussed by 
Bromme and Gierth in chapter 14.3 of this handbook. 

A different topic is communicative rationality. In subsec- 
tion 7.2 of this introductory chapter, we briefly discussed 
the extent to which rationality presupposes linguistic 
faculties. A complementary idea is that social linguistic 
behavior, that is, communication, embodies a particular 
form of rationality. A prominent example is the approach 
developed in the opus magnum of Habermas (1981), who 
has a democratic notion of practical rationality that tran- 
scends that of instrumental rationality. For him, rationality 
is rather a form of public justification. Thus, communica- 
tive rationality is manifested in a discourse free of domi- 
nation, in which all arguments are considered without 
prejudice. Rationality and validity claims in general have 
to prove themselves in such a discourse. Ultimately, com- 
municative rationality is guided by a very Kantian idea: 
namely, that in rational discourse, we should treat our 
fellow humans not as means to our ends but as ends in 
themselves. If so, communicative rationality would be 
entailed by one version of Kant’s categorical imperative. 

Not surprisingly, Habermas enthusiastically received the 
program of inferentialist semantics consistently expounded 
by Robert Brandom (1994, 2000). There, a linguistic com- 
munity is represented as a community of reason-givers 
and reason-takers, and it is explained how this exchange 
of reasons and inferential relations is constitutive of lin- 
guistic meanings. Yet, Brandom’s notion of a reason is 
based on the ordinary notion of an epistemic or practical 
reason, as it is widely discussed in this handbook (e.g., in 
chapter 2.1 by Broome and chapter 5.3 by Kern-Isberner, 
Skovgaard-Olsen, & Spohn, both in this handbook). So 
Brandom does not support the assumption of a particu- 
lar communicative rationality. In fact, he only insists that 
linguistic practice is an essentially rational practice. 

A different account of communicative rationality is pro- 
vided by H. P. Grice’s (1975, 1989) theory of conversational 
implicatures. This theory conceives of communication 
as a rational activity governed by the so-called coopera- 
tive principle. Based on this principle, Grice shows why 
communication works the way it does. However, nothing 
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more than the rationality of cooperation is assumed in 
his approach, and again no deeper theory of rationality 
is involved. 

Indeed, in analytic philosophy, there is a general ten- 
dency to deny a special form of communicative rational- 
ity. This tendency is clearly expressed in chapter 10.3 by 
Meggle (this handbook). It mainly attempts to develop 
the intentionalistic account of meaning of Grice (1957), 
which, in contrast to Habermas, does not assume richer 
forms of rationality transcending instrumental rational- 
ity. David Lewis (1969) then builds on Grice’s approach 
and amends it by his account of conventions in general 
and meaning-constitutive linguistic conventions in par- 
ticular. This account appeals to game-theoretic rational- 
ity, which Meggle conceives as a form of instrumental 
rationality and not as a more embracive kind of practical 
rationality, as Habermas sees communicative rationality. 
However, we can interpret the game-theoretic approach 
also in a different way and argue that according to game 
theory, individuals treat each other as persons with their 
own aims and interests and thus respect the Kantian for- 
mula. If so, the analysis of conventions and linguistic 
communication provided by this approach conforms to 
Habermas’s ideal as well. 

These are just three examples that illustrate that the 
forms of rationality that play a role in communication 
are already well covered in this handbook. They also 
explain why we did not further pursue the prospect of 
a special theory of communicative rationality. This also 
conforms to our general attitude of not dealing with 
philosophical and psychological issues of concept for- 
mation and linguistic meaning in this handbook. 

A very different issue is the relation between ratio- 
nality and artificial intelligence (Al). In section 7 of this 
introductory chapter, we already learned that rational- 
ity should not be identified with intelligence. Hence, we 
will not discuss here whether AI systems are intelligent. 
But we should discuss whether such systems are ratio- 
nal in the general sense or whether we need a special 
account of rationality to evaluate AI systems. To do this, 
it is important to distinguish two different approaches 
to artificial intelligence: symbolic Al, in which knowl- 
edge and reasoning procedures are explicitly represented 
by symbols, constraints, and predicates, and AI based 
on machine learning, which uses statistical methods to 
identify patterns and regularities in massive data sets 
to make predictions and decisions. Interestingly, the 
two approaches have opposite strengths when it comes 
to epistemic and practical rationality. In symbolic AI 
systems, all information is explicitly represented in 
the “knowledge base” and is more or less readable for 
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humans. Importantly, it is required to keep the knowl- 
edge base consistent, and where inconsistencies appear, 
rational updating and revision strategies are used to rees- 
tablish consistency. Consistency is one of the general 
requirements of epistemic rationality and a research area 
where AI, philosophy, and cognitive psychology are inti- 
mately linked. In fact, many theories of rational belief 
revision were developed at the intersection of these dis- 
ciplines (Gardenfors, 1992) and are already represented 
in the general concept of epistemic rationality. 

The epistemic rationality of symbolic AI systems is 
not limited to the way knowledge is represented but 
concerns also the way inferences are performed. Typi- 
cally, symbolic AI systems use the general principles of 
classical logic to draw inferences and evaluate conclu- 
sions. However, this approach has many difficulties and 
thus has been complemented by systems of nonmono- 
tonic logic, which mirror different, sometimes weaker, 
rationality requirements. Still, the goal of symbolic Al is 
to build systems that resemble or even exceed human 
epistemic rationality. 

The problem, though, is that even if these systems 
reach a relatively high level of epistemic rationality, 
they often fail to show satisfactory performance when it 
comes to decision making. For instance, so-called expert 
systems, which boomed in the 1980s, were not very 
good at making decisions in medicine, law, or other 
domains. To understand the limits of these systems, we 
did not need a special conception of AI rationality. 

The limited practical rationality of symbolic AI systems 
is probably one reason why, in areas where decisions are 
concerned, they have been largely ousted by AI systems 
based on machine learning. These methods are not new; 
they were already used in the 1980s, but with limited 
success (Rumelhart, McClelland, & the PDP Research 
Group, 1986). Today, AI systems based on machine 
learning perform much better, mainly due to the enor- 
mously increased computing power of computers and 
the huge amounts of data from the Internet and social 
media. The AI systems are trained with these giant data 
sets and learn to recognize statistical patterns, which are 
then applied to new data. The training makes it possible 
to make decisions, for example, which costumer receives 
a loan, which stocks are bought or sold, for which patient 
a certain medical therapy is still worthwhile or not, how 
new employees are hired, and so on. Many of these sys- 
tems do not perform as well as Silicon Valley wants us to 
believe. Still, we can evaluate their outputs with our gen- 
eral understanding of what is a good decision. 

The problem, though, is that nobody can understand 
how such systems reach their decisions. We can only 
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evaluate the output of the system but not the processes 
that led to its decision. Moreover, such systems have 
very limited epistemic rationality. They rely on massive 
data sets but not on knowledge in the sense of true and 
justified beliefs. They cannot explain how they acquire a 
belief, which we usually see as a requirement for rational- 
ity in humans. An alternative might be explainable artifi- 
cial intelligence, which combines machine learning and 
symbolic AI to make its knowledge and decisions more 
transparent. Such systems might have the potential to 
exhibit more theoretical and practical rationality—in the 
general sense of these concepts. 


8.2 Overlaps with Rationality 

There is hardly any situation in which our beliefs and 
actions are driven, or should be evaluated, solely by the 
norms of rationality. We are of course aware that ratio- 
nality issues strongly overlap with many other concepts 
and facets of human mental life that we have not yet 
covered in this introductory chapter. So, let us at least 
briefly address some relations to other topics in the 
vicinity of rationality. 

One such topic is the relation between rational- 
ity and emotions. It surfaces a little in some chapters 
(chapter 8.1 by Griine-Yanoff; chapter 8.6 by Thomp- 
son, Elqayam, & Ackerman; and chapter 9.4 by Dhami 
& al-Nowaihi), but it is nowhere systematically treated 
in this handbook. This may be surprising for psycholo- 
gists, as the general relationship between emotion and 
cognition is an issue with enormous theoretical and 
practical relevance (Damasio, 1994). The situation looks 
quite different, however, if we look at the more specific 
relationship between rationality and emotions. On the 
one hand, neuropsychological studies with patients 
show that people who are incapable of experiencing 
emotions but have retained their cognitive abilities can 
be seriously impaired in certain decision-making tasks 
(Dimitrov, Phipps, Zahn, & Grafman, 1999). On the 
other hand, psychological laypeople and experts tend 
to assume that a “cool head” thinks better than a “hot 
head” (Blanchette, 2013). Only recently has this topic 
been systematically investigated in the field of logical 
thinking. Several questions have played a role here: one 
is whether the current affective state of a person influ- 
ences her ability to think logically, independently of 
the content of the reasoning task. Another question is 
whether the emotional content of a reasoning problem 
itself influences logical reasoning performance. And, of 
course, we can ask how the emotional content of the 
problem and the person’s affective state interact with 
each other. 
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The findings are still equivocal. Some empirical stud- 
ies show that people who witnessed the terror attack in 
London on July 7, 2005, at close range and who thus had 
strong emotions performed better than other people on 
logical reasoning problems with contents related to ter- 
ror (Blanchette, Richards, Melnyk, & Lavda, 2007). But 
laboratory studies show that both positive and negative 
emotional states of the reasoner can have a devastating 
effect on reasoning performance (Jung, Wranke, Ham- 
burger, & Knauff, 2014). And interactions between the 
emotional value of a content and the emotional state 
of the reasoner have seldom been reported (Blanchette, 
Caparos, & Trémoliére, 2018). We cannot discuss all these 
interesting findings, although it is likely that inconsis- 
tent results might also have to do with the fact that for 
ethical reasons, strong emotions cannot be induced in 
laboratory research. Moreover, the many different kinds 
of long-term and short-term affects, sentiments, feel- 
ings, emotions, or moods may interact with rational rea- 
soning in many different ways. They can, for instance, 
put additional load on the cognitive system, but can 
also be adaptive and help to avoid hazardous situations 
(De Jong, Mayer, & Van Den Hout, 1997). Already in the 
1960s, Schachter and Singer (1962) argued that the expe- 
rience of emotion consists of two components: physical 
arousal and its cognitive interpretation. In other words, 
when an emotion is felt, the person is in a particular 
physiological state and must figure out what caused this 
arousal. Since our bodily states are highly ambiguous, 
emotion itself requires a cognitive act of interpretation 
and thus of reasoning (Chater, 2018). 

This leads directly to the main question for philoso- 
phers: can emotions, too, be normatively assessed with 
regard to their rationality? In subsection 2.2 of this intro- 
ductory chapter, we put this question aside and focused 
on actions and beliefs as the sole objects of rationality 
assessments. However, one might also have the view that 
emotions do not only have causes, do not just befall us, 
but also have reasons and are amenable to reasons. We 
all are familiar with the fact that one is not automati- 
cally exculpated by saying, “Sorry, but this is how I feel.” 
People can have more or less “adequate” or, conversely, 
exaggerated or even pathogenic feelings and emotional 
responses. They can have persistent emotions whose 
cause has long dissolved. There can be an absence (or 
suppression) of feelings where one would expect them to 
show, and so on. However, even if we grant that emotions 
are responsive to reasons, one may doubt that assessing 
the adequacy of emotions is already a judgment about 
their rationality. The topic is richly discussed (see, e.g., 
de Sousa, 2011; D. Evans & Cruse, 2004; Helm, 2001). 
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Again, we have preferred not to cover this topic in the 
handbook. 

A connected topic is the relation between rationality 
and morality. Here, emotions are involved in two ways. 
The immediate connection is pursued by sentimentalist 
theories of morality, which we find, for instance, in the 
work of Hume (1751/1975a) and Schopenhauer (1841). 
Clearly, we have particularly moral sentiments: positive 
ones like love, sympathy, and compassion and negative 
ones like remorse, shame, blame, and outrage. For sen- 
timentalist theories, these moral feelings constitute the 
origin of morality and lie at the basis for justifying moral 
judgments. 

However, there is also an indirect connection. Even 
if one does not appeal to particular moral sentiments, 
one may locate the basis of all our evaluations in how 
we feel. We ultimately strive for happiness, and we seek 
pleasure and try to avoid pain—this is the traditional ter- 
minology, which had a broader meaning than it does in 
current everyday language. In this picture, our ultimate 
values refer to certain positive or negative emotional 
or hedonic states, and then morality is about the gen- 
eral realization of these values. Ancient hedonism refers 
to pleasures and pains, and eudaimonism to happiness 
(though eudaimonia was not conceived as just a psycho- 
logical state). We find it in Jeremy Bentham’s (1789/1970) 
hedonic utilitarianism. And we find it in contractarianism, 
originating with Hobbes (1651/1994), where the social 
contract enforcing moral behavior guarantees the indi- 
vidual pursuit of happiness and the avoidance of mutual 
harm. These moral conceptions were driven by the idea 
that morality can be explained by the rational pursuit 
of these ultimate values. This would be a strong justi- 
fication: morality would just be instrumental to those 
values. 

Kant (1788/1908), of course, is famous for being 
opposed to all of this. For him, the first principle of 
morality, the categorical imperative, is an a priori principle 
of pure reason and thus derives from rationality alone, 
without appealing to pains, pleasures, or other feelings. 
This obviously presupposes a stronger notion of ratio- 
nality, transcending Hume’s instrumental rationality 
(see also chapter 1.1 by Sturm, this handbook). Again, 
these issues are much too large to be treated properly in 
this handbook, but they are reflected in chapter 12.1 by 
Fehige and Wessels and chapter 12.2 by Smith (both in 
this handbook). 

While philosophers reflect on the nature of moral 
sentiments and their relation to rationality, cognitive 
psychology and neuroscience conduct experiments to 
understand how moral values interact with other cog- 
nitive and noncognitive processes. One active research 
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area in this context is law and legal reasoning. Do people 
use moral values or legal norms when thinking about 
crimes and court decisions? Empirical studies show that 
legal experts such as judges and lawyers can follow the 
rational norms of the legal system, whereas laypeople 
often base their opinions about crimes and how they 
should be punished on their intuitive feeling of right- 
ness and morality. In particular, if they judge a crime to 
be exceptionally immoral and evil, they tend to demand 
very severe punishments. And they do this even if, from 
a legal point of view, there are mitigating or even excul- 
patory conditions, just to satisfy their need to punish the 
transgressor (Gazzo Castafieda & Knauff, 2016). Legal 
logic and logical models of legal argumentation are the 
topics of two quite different chapters in this handbook: 
chapter 11.3 by Hilgendorf and chapter 11.4 by Prakken. 

Another line of experimental research actually goes 
back to a thought experiment by Karl Engisch, a Ger- 
man jurist and philosopher of law who considered in his 
study (1930, p. 288) what is nowadays called the “trolley 
problem”: imagine you are standing on a footbridge and 
see a runaway trolley moving toward five people lying 
on the main track. You are standing next to a lever that 
controls a switch. If you pull the lever, the trolley will 
be redirected onto a side track and the five people on 
the main track will be saved. However, there is a single 
person lying on the side track. You have two options: do 
nothing and allow the trolley to kill the five people on 
the main track, or pull the lever and cause the trolley 
to kill the single person on the side track. Is one option 
morally required? 

This thought experiment has been much discussed 
in philosophy to differentiate between deontological 
and consequentialist, in particular utilitarian, accounts 
of morality. Utilitarians would recommend pulling the 
lever because it results in fewer fatalities, while deontol- 
ogists would object that human life is an incommensu- 
rable good, so that counting fatalities is not necessarily 
an argument. Instead, they would argue that it is better 
to refrain from pulling the lever, and thus to let a greater 
number die, than to actively do something resulting in 
the death of (fewer) persons. 

Psychologists, by contrast, investigate how people 
actually (say they would) decide. They also compare this 
scenario to a slightly different situation: imagine that a 
fat man is standing next to you, and instead of turning 
a lever, you could push him on the track, thereby again 
saving the group of five but killing the fat man. How do 
people decide now? 

Most of the empirical results show that persons decide 
differently in the two situations (Awad, Dsouza, Sha- 
riff, Rahwan, & Bonnefon, 2020). Is that rational? A PhD 
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student of philosophy, Joshua Greene, was the first who 
used this dilemma in a functional brain-imaging study. 
Greene, Sommerville, Nystrom, Darley, and Cohen (2001) 
showed that the “personal” dilemma (pushing the man 
off the footbridge) activated brain regions associated with 
emotions, while the “impersonal” dilemma (flipping a 
lever) engages regions associated with controlled, rational 
reasoning. These findings (and their interpretations) are 
problematic for many reasons (Waldmann, Nagel, & Wieg- 
mann, 2012) but are related to many different versions of 
dual-process accounts of moral decision making, which 
are still popular in cognitive research (see chapter 2.5 by 
Klauer, this handbook). Again, the topic of morality is too 
comprehensive to receive treatment in this handbook, but 
chapter 12.3 by Wiegmann and Sauer (in this handbook) 
comments on it from the psychological side. 

By the way, it may be unsurprising that moral 
research today gets much attention from military and 
engineering fields, for instance, in the development of 
autonomous weapons or vehicles, for which situations 
can occur where one or another potentially fatal colli- 
sion is unavoidable. In such cases, the car’s software has 
to “decide” what to crash into and which casualties to 
hazard. In 2018, the moral machine project began to col- 
lect information on such decisions on an Internet plat- 
form, where people can choose between two different 
destructive outcomes (Awad et al., 2018). This looks like 
a dangerous blurring of the border between empirical 
decisions and normative theories, which we try so care- 
fully to reflect in this handbook. Since Hume, we know 
that we better avoid inferences from is to ought. 

We may widen the perspective even further, from the 
relation between morality and rationality to the general 
cultural, social, and political embedding of rationality. Since 
Plato, political philosophy has been deeply involved 
in rationality issues. In the Republic, he described how 
society should be rationally organized: ruled by a lead- 
ing class that is rational, intelligent, and self-controlled 
(Plato, 2013). Hobbes, in the Leviathan (1651/1994), 
argued that citizens are only protected from mutual 
violence, as it occurs in the state of nature, if power is 
delegated to a reasonable sovereign. Later, Jean-Jacques 
Rousseau, in The Social Contract (1762), challenged this 
view by arguing that all members of a society have the 
same rights and duties. Although these issues are super- 
imposed by more important topics of political philoso- 
phy, such as liberty, equality, and justice, they also have 
much to do with rationality, yet could not be covered in 
this handbook. The lack becomes particularly apparent 
when we consider John Rawls’s (1971) theory of justice as 
fairness, in which the so-called veil of ignorance creates 
a situation in which rational decision making arguably 


49 


justifies his principles of justice. But then those princi- 
ples are at issue, not the theory of rational decision 
making. Hence, we preferred to abstain from extending 
the handbook in this direction. It is reflected, to some 
extent, in chapter 10.5 by Nida-Rimelin, Gutwald, and 
Zuber (this handbook). 

We have also abstained from engaging in intercul- 
tural considerations. A review of the sparse literature 
that is available in languages we understand suggests 
that there is a wide range of alternatives to the con- 
ception of rationality in the European and Northern 
American intellectual tradition. For instance, a core dis- 
tinction in this handbook is that between normative 
and descriptive views on rationality. The distinction is 
so deeply entrenched in our scientific thinking that we 
can hardly imagine doing without it. But is this neces- 
sary? Perhaps not. For instance, the distinction does 
not seem to be clearly present in traditional Chinese 
thought. Instead of this dichotomy, we find a general 
tendency in traditional China to think of things as inter- 
actions of yin and yang and the more fluid interactions 
of the “five phases,” water, wood, fire, metal, and earth 
(Marchal & Wenzel, 2017). Comparative ethnologists 
have found various other examples of fundamentally 
different ways of thinking: people in Melanesia and 
Polynesia use numerical systems that significantly dif- 
fer from the norms of Western mathematics (Beller & 
Bender, 2008). Islam provides alternatives to the eco- 
nomic rationality of rational choice theory (Tafer, Bous- 
sahmine, & Bouanini, 2016), and studies looking at the 
brains of people playing a fairness game found very 
different neural activities in Buddhist meditators and 
Western participants (Kirk, Downar, & Montague, 2011). 
Such observations challenge our arrogance that rational- 
ity everywhere can be judged by a single set of criteria. 
It is not unlikely that we find diverging conceptions of 
rationality in different cultures and different socioeco- 
nomic systems (Lloyd, 2017). Relatedly, Henrich, Heine, 
and Norenzayan (2010) have argued that psychologists 
and other behavioral scientists often make strong claims 
based on experiments with samples drawn entirely from 
Western, Educated, Industrialized, Rich, and Democratic 
(WEIRD) societies. However, we confess that the present 
handbook is committed to the Western intellectual tra- 
dition in psychology and philosophy. 

This is the right place to finally explain why so many 
areas of so-called continental philosophy are not repre- 
sented in this handbook. We do not want to suggest that 
they have nothing substantial to say about rationality. 
In fact, reflections on reasons and rationality are ubiq- 
uitous also in these parts of philosophy. For instance, 
consistency is central to most concepts of rationality, 
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but in dialectic thinking, contradictory hypotheses lead 
to a rational synthesis. Dialectic thinking is important in 
the work of Georg Friedrich Wilhelm Hegel but not rep- 
resented in this handbook. Friedrich Nietzsche is famous 
for his perspectivism (see, e.g., R. L. Anderson, 1998; 
Danto, 1965, chapter 3). His criticism was that scholars 
are often not aware of their own perspective and thus 
do not control its influence on their work. Maybe his 
demand for perspectival deconstruction should be part 
of epistemic rationality, but it is neither treated in this 
handbook nor exercised by its authors. Or think of the 
profound criticism of Western technical rationality in 
Martin Heidegger (1954). It may be misguided, but it 
springs from deep philosophical sources and is still dis- 
concerting. Michel Foucault has tremendously shaped 
our intellectual landscape. The power to delimit reason 
is key to his early analysis of madness (Foucault, 1972), 
and his later analysis of government shows the historical 
conditions of the possibility of different types of govern- 
mental rationalities (Foucault, 2008). And when Haber- 
mas is discussed under the heading of communicative 
rationality in chapter 10.3 by Meggle (this handbook), 
this is restricted to the narrow perspective of philosophy 
of language and neglects Habermas’s much more general 
political intentions concerning the theory of democracy. 

So why is this handbook so obviously biased against 
continental philosophy and toward analytic philosophy, 
and within the latter toward the more formal ways of 
philosophizing? Before we explain this bias, we should 
mention that already the alleged opposition between 
“analytic” and “continental” is infelicitous, as “ana- 
lytic” vaguely designates certain ways of philosophizing, 
which, however, still diverge dramatically, and “con- 
tinental” refers to geographical locations and origins, 
which again have produced quite different strands and 
styles of philosophizing. 

But even if we follow this traditional terminology, 
there are several reasons why this handbook is biased 
toward analytic philosophy. One reason is that the hand- 
book brings together cognitive psychology and philoso- 
phy, and that analytic philosophy lies much closer to 
the intersection of these disciplines than other parts of 
philosophy. This is why analytic philosophers and cog- 
nitive psychologists collaborate so successfully under 
the roof of cognitive science. For the same reason, we 
also have a certain bias toward the more formal side of 
analytic philosophy, which provides more specific and 
detailed accounts of the norms of rationality and has 
more natural connections to cognitive theories of the 
human mind. Another, even more fundamental reason 
for our bias is that continental philosophy often focuses 
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on the wider cultural and political context of rationality, 
as our examples above indicate. However, as just said, 
this handbook is silent on this wider context. Although 
such perspectives on rationality are interesting and 
important, they do not fit well with the predominantly 
individualistic orientation of psychological research and 
are difficult to investigate with the methods of experi- 
mental psychology. Analytic philosophers have similar 
reservations. For them, such political and cultural asso- 
ciations are often sweeping and hard to seize—none of 
these accounts could be subject to sophisticated norma- 
tive discourse, as we find it in analytic philosophy. So, 
we had to draw a line here, and therefore the contribu- 
tions of continental philosophy are not represented in 
this handbook. 


9. Conclusion and Open Issues 


The aim of this chapter was to offer the reader a guide 
through the complex field of rationality research. We 
hope that we could make clear why we consider the dis- 
tinction between theoretical and practical rationality so 
important. We also emphasized why the relation between 
normative and descriptive theories of rationality is so 
essential for our philosophico-psychological enterprise 
and why research on social rationality should go beyond 
competitive scenarios in which the actors consider only 
their individual benefits. We moreover explained why 
it is important that rationality research should focus 
not only on the rational assessment of the results of a 
thought process but also on the cognitive processes that 
led from the input to the output. Furthermore, we intro- 
duced the concept of a double equilibrium, which requires 
empirical research to listen to normative theorizing and, 
conversely, reminds us that normative considerations are 
always defeasible and must withstand critical, empirical 
scrutiny. In the final sections, we have discussed some 
cognitive prerequisites of rationality and argued against 
the view that many specialized conceptions of rational- 
ity are needed for the many areas of everyday life, public 
discourse, and the sciences and humanities. Our position 
was that rationality assessments in most of these areas 
can rely on general concepts of rationality. 

Of course, there are many open questions and tasks 
that rationality research still has to tackle. One of these 
tasks is to overcome the fixation on logic and probability 
theory, which is particularly prevalent in the psychol- 
ogy of reasoning. We have shown that philosophy has 
for decades offered a richer set of options. Psychology 
has just started to pay more attention to such accounts, 
and we hope that it will continue to go in that direction. 
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While this is an issue of epistemic rationality, we see a 
related challenge in practical rationality. Here, we think 
that it is important to further develop alternatives to 
standard decision theory. Again, we believe that psy- 
chology and philosophy should go beyond these limita- 
tions of present research. There are developments in this 
direction, but currently it seems that philosophy and 
psychology are pursuing different lines. More integra- 
tion would be highly desirable. 

A related observation is that most of the current work 
under the heading of social rationality is in fact about 
individual rationality in social situations. We think this is 
a weakness of current research that should be overcome. 
In so many situations of everyday life, people seek to 
think and act rationally in concert with others, but we 
do not know much about the underlying cognitive pro- 
cesses. This is a research field where social psychologists, 
cognitive psychologists, social scientists, and philoso- 
phers should collaborate more intensively. We also do 
not know enough about the normative standards that we 
should use to assess collective forms of rationality, which 
is another area where we think that rationality research 
must make progress. This is important for many funda- 
mental questions of rationality, but it is obvious that this 
topic is also socially and politically highly relevant. 

Another characteristic of current research is that in 
the various subfields, the process-oriented perspective is 
pursued sometimes more and sometimes less intensely. 
Surely this is partly due to the varying difficulties of 
doing so. Some theories from philosophy may be really 
hard to bring into process-oriented terms. Still, we think 
that this is desirable and would strengthen rationality 
research where it is yet weak. 

We also think that philosophers and psychologists 
should try to reach more consensus on the nature of con- 
cepts and meaning. Currently, there seems to be much 
dissent on this topic between the two disciplines, which 
affects many areas of cognitive science. For the same rea- 
son, we did not discuss the extremely complex question 
of what we mean when we talk about truth. Truth may 
be the ultimate goal of rationality, but there are so many 
different conceptions of truth in philosophy and psy- 
chology that it is hard to see how a more unified view 
could be reached in the near future. Fortunately, this is 
not primarily a disagreement about rationality. How- 
ever, in the long run, more collaboration on the topics 
of meaning and truth between the different disciplines 
would certainly also be beneficial for our understanding 
of rationality. 

All these issues for future research need much 
more interdisciplinary interaction. We think that this 
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handbook can show that by such interdisciplinary col- 
laboration, research into rationality has already made 
great progress. Today we have much more knowledge 
about human rationality than each of our disciplines 
could have achieved on their own. However, there is 
still room for further progress. For instance, it is essen- 
tial that all parties in rationality research be clear about 
the complex relation between descriptive and norma- 
tive considerations. Unclarity about this relation still 
seems to hamper research. And we think that other dis- 
ciplines should be more involved in our interdisciplin- 
ary endeavor. Linguists, for instance, have substantial 
things to say about rationality in language, pedagogues 
and education researchers should systematically study 
how students can be trained in rational thinking, and 
anthropologists and ethnologists can help us better 
understand how rationality is rooted in different cul- 
tures and human lifestyles. 

We began this introductory chapter with the Aris- 
totelian definition of humans as rational animals, and 
we mentioned a few objections to this view. So, are we 
really as rational and smart as we think? Or is this more 
a self-delusion that persists even if empirical findings 
put our optimism into question? The intention of this 
chapter was not to answer this question. That would be 
presumptuous. What we wanted to do, rather, was to 
help our readers to develop their own opinion on this 
question. This opinion, we hope, might benefit from the 
interdisciplinary and multiperspectival character of this 
handbook. In the end, we may agree with how Christof 
Koch (2016, p. 25) summarized the lessons from Dar- 
win’s theory of evolution: we are unique, but so is every 
other species, each in its own way. 


Notes 


1. There are many labels for this and similar distinctions: “is— 


mu 


ought,” “ontic-deontic,” “descriptive—prescriptive,” “empirical 
—normative.” We have decided to use the pair “descriptive— 
normative” throughout and thus follow the usage of Elqayam 


and Evans (2011). 


2. The Priority Programme “New Frameworks of Rational- 
ity” was an important step toward improved collaboration 
between psychologists, philosophers, and researchers in other 
disciplines in the field of rationality (see the Preface in this 
handbook). 


3. For all this, see the entries “Ratio,” “Rationalitat,” and “Ver- 
nunft, Verstand” in Ritter, Griinder, and Gabriel (1971-2007). 


4. Section 2 of chapter 2.1 by Broome (this handbook) also 
contains detailed remarks about the etymology of the English 
word “reason.” 
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5. The other two are his famous context principle and his rule 
to strictly distinguish object and concept. 


6. Knauff and Gazzo Castafieda (2021) criticize this terminol- 
ogy and show that the new trend toward subjective degrees of 
belief cannot be regarded as a “paradigm shift” in the Kuhnian 
sense. 


7. Note, though, that the label “Bayesianism” can take on 
quite different meanings in psychology (Jones & Love, 2011). 


8. A philosopher would say that we should talk of “doxastic” 
attitudes. Doxastic attitudes are belief attitudes in all shades 
and grades: opinions, doubts, subjective probabilities, and so 
on, while epistemic attitudes, strictly speaking, concern only 
knowledge. Still, “epistemic attitude” is the more familiar term. 
We shall therefore use it always in the wide sense of “doxastic 
attitude.” Of course, the philosopher would add that this simply 
conforms to the sloppy talk about knowledge found throughout 
the sciences, which hardly distinguishes knowledge from true 
or certain belief. And he would refer to the huge philosophical 
discussion of knowledge in the past 50 years, which is largely 
neglected by those sciences (see, e.g., Bernecker & Pritchard, 
2011, particularly parts I-IV and VII; or Williamson, 2000), 
as the main advocate of the opinion that the study of knowl- 
edge even precedes the study of belief). Let this remark suffice 
to indicate that the concept of knowledge is a philosophical 
snakepit from which we keep a healthy distance despite our talk 
of epistemic attitudes. 


9. After the heyday of so-called logical behaviorists like Ryle, 
who subsumed action explanations under the special kind of 
(noncausal) dispositional explanation, the view that actions 
are caused by intentions in an ordinary way was initiated 
by Hempel (1961-1962) and Davidson (1963) and is still the 
dominant view in philosophy (see, e.g., Bratman, 2006). 


10. The following considerations are more extensively explained 
in Spohn (2002). 


11. Many philosophers say that beliefs are rational relative to 
the facts, or that only facts can be reasons for beliefs. But then 
these facts must be perceived and hence believed, and so this 
relation is again mediated by beliefs. 


12. This may be taken as a modern version of Berkeley’s 
dictum that only ideas can be similar to ideas (cf. Berkeley, 
1710/1949, section 8). 


13. The situation resembles the Agrippa trilemma, often taken 
to be the basic problem of epistemology (see, e.g., BonJour, 
1985), according to which a belief is justified only if it is jus- 
tified by reasons that are in turn justified. The trilemma has 
a skeptical force: how, then, could a belief ever be justified? 
Many versions of foundationalism and coherentism, justifica- 
tion internalism, reliabilism, and contextualism respond to this 
challenge. We need not discuss this here. Certainly, though, 
the situation vis-a-vis reasonableness is similar. 
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14. Spohn (2002) expands on these workings of rationality 
assessments. 


15. Elqayam and Evans (2011, p. 238) similarly speak of the 
“product” and the “process level.” 


16. In philosophy, the debate about the nature of those con- 
tents, as well as about the nature of the concepts of which they 
consist, is endless, most sophisticated, and without conclusion 
(see, e.g., Fodor, 1998; Garcia-Carpintero & Macia, 2006). 


17. In fact, the formal structure of probabilities was not firmly 
fixed from the beginning; see Shafer (1978). 


18. A plea for epistemic dualism is found in Spohn (2012, 
chapter 10); one of several attempts for unification is found 
in Leitgeb (2017). For an attempt at connecting the issue with 
psychological research, see also Weisberg (2020). 


19. The conception of intentional objects goes back to Bren- 
tano (1874). The book that alerted philosophers most to the 
problems surrounding intentional objects, propositions, and 
the like was Quine (1960). The subsequent discussion is end- 
less, without a clear conclusion. For a recent treatment, see 
Recanati (2016). See also the plea for intentional objects in 
Spohn (2009, chapter 16). 


20. The basic algorithm is the so-called roll-back analysis as 
explained in Raiffa (1968). 


21. Often, though, the issue is not individual rationality 
but rather policy: how to design institutions when these are 
assumed to have an influence on people’s preferences? 


22. There is no textbook summarizing all these alternative 
accounts. One may consult the comprehensive handbook by 
Glanzberg (2018). The richest, though critical, discussion of 
antirealistic truth theories in analytic philosophy is found in 
Devitt (1991). 


23. Such thoughts are also vigorously discussed in philoso- 
phy, based on the brain-in-a-vat scenario of Putnam (1981, 
chapter 1). 


24. The term was introduced by C. I. Lewis (1912). 


25. Grice (1975) tried to save this representation with the help 
of his theory of implicatures; see also Bennett (2003, chapter 2). 


26. It was a central problem for logical empiricism to explain 
how we can understand not directly observable dispositions 
on the basis of observational language and extensional logic. 
(Philosophers have a more general understanding of disposi- 
tions than psychologists; see also section 6 of this introductory 
chapter.) This problem turned out to be a Kuhnian anomaly 
for logical empiricism and eventually led to its breakdown. 
The problem was precisely how to understand condition- 
als like “If that piece of sugar were put into water, it would 
dissolve” (=“That piece of sugar is soluble”) in terms of clas- 
sical logic and material implication. Compare, for example, 
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Hempel’s paradigmatic movement from Hempel (1958) to 
Hempel (1973). 


27. So, this was at the same time when the Wason selection 
task was designed. A different attempt to cope with these 
paradoxes is relevance logic, developed by A. R. Anderson and 
Belnap (1975). It is, however, not represented in this handbook. 


28. Cohen (1981) gives a similar description, although he dis- 
tinguishes between a narrow and a wide reflective equilibrium. 


29. It is not only a methodological issue. Philosophers tend to 
carry it deeply into metaphysics and semantics, usually within 
metaethics and with respect to morality and moral predicates 
and properties. However, with respect to normativity, morality 
and rationality are in the same boat. Thus, these philosophi- 
cal discussions, as pursued, for example, in Ridge (2019), carry 
over to rationality. 


30. Hume (1975c, p. 469) famously said, referring to the transi- 
tion from is to ought, that it “seems altogether inconceivable, 
how this new relation can be a deduction from others, which 
are entirely different from it.” The label “naturalistic fallacy” 
was introduced by Moore (1903, §10) for the related fallacy of 
inferring the goodness of something solely from its descriptive 
properties. 


31. Paradigmatically by Hempel (1961-1962). 


32. To our knowledge, this kind of double reflective equi- 
librium was first described in Spohn (1993, p. 188). See also 
Spohn (2002, section 4). 


33. As is more fully argued in Spohn (2011). 


34. Rich textbooks are, for example, Myerson (1991) and Masch- 
ler, Solan, and Zamir (2013). See also section 9.1 by Albert and 
Kliemt (this handbook). 


35. The distinction between social rationality and individual 
rationality we are invoking here is more salient in the practi- 
cal context. That is why only this context is considered here. 
However, it pertains just as well to the epistemic context: see 
the distinction between epistemology in groups versus episte- 
mology of groups in chapter 10.1 by Dietrich and Spiekermann 
(this handbook). 


36. There are, however, many other paradigmatic and highly 
interesting social situations modeled by game theory. For 
brief explanations, see chapter 9.1 by Albert and Kliemt (this 
handbook). 


37. It has been important to logical behaviorists like Ryle 
(1949). For a general discussion of the philosophical intricacies 
of dispositions, see, for example, Mumford (1998). 


38. One could argue that the so-called replication crisis (that 
many published findings could not be replicated in subsequent 
studies) is a drastic example of the limits of epistemic rationality 
in science (Open Science Collaboration, 2015). However, many 
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researchers have questioned the popular interpretation of the 
relatively low rates of replicability. For example, a “regression to 
the mean” can explain why even if the initial results were cor- 
rect, they need not be replicable in subsequent studies (Fiedler 
& Prager, 2018). The publication from 2015 also showed that 
replicability varies strongly over different subdisciplines and 
that replication rates in cognitive psychology are better than in 
other areas. 
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Part! Origins and Key Issues of Rationality 


Section 1 Origins of Rationality 


1.1 


Thomas Sturm 


Summary 


This chapter considers the history of the descriptive— 
normative divide, viewed as fundamental to, but also 
challenging, current research on rationality. I show how, 
from antiquity through early modernity, this distinc- 
tion never became a topic of discussion. That changed, 
however, during the Enlightenment, through a mixture 
of new metaphysical and scientific ideas, and, impor- 
tantly, due to philosophical struggles over the nature, 
potential, and limits of rationality. The distinction first 
became explicit in moral philosophy through Hume’s 
“js-ought distinction.” In Kant, we find more sophis- 
ticated uses of it in both his theoretical and practical 
philosophy. Criticisms of those views are also discussed, 
with an eye toward examining how they are related to 
competing concepts of rationality. Finally, I show how 
this distinction became emphasized but also criticized in 
the psychologism debate and still shapes discussions in 
philosophy and psychology today. 


1. Three Historical Stages of the Distinction 
“Reason”! or “rationality” has been called an “accordion 
term” (Burian, 1977): its meaning is highly ambiguous 
(Black, 1986), “fragmented” (Evans, 1991; Stich, 1990), 
or hard to give a unified account of (Audi, 2001). A his- 
torical review proves this to be true. Plato’s logos differs 
substantially from Descartes’s ratio or intellectus, both 
differ from Hume’s notion of reason as well as from 
Kant’s complex concept of Vernunft, and so on. The 
Greek, Latin, French, German, and English terms logos 
and nous, ratio and intellectus, raison and entendement, 
Vernunft versus Verstand, or “reason” and “understand- 
ing” (or “intellect”) are related but different and at times 
even at odds (Rapp et al., 2001). Again, the disciplines 
dealing with rationality—philosophy, statistics, and the 
cognitive and social sciences—have different agendas 
and therefore use different concepts. It is common to 
say, for instance, that philosophy deals with rationality 
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only from a normative point of view, whereas psychol- 
ogy is merely descriptive. Historical inquiry can reveal 
how this view came into existence but also why it is his- 
torically as well as philosophically problematic. 

In the following analysis, I focus on a number of exem- 
plary authors writing up until the early 20th century. In 
doing so, I concentrate on the concept of rationality (and 
theories in which it figures) in relation to the descriptive— 
normative distinction instead of mere terminology or 
etymology. Also, the discussion will be problem centered: 
When, where, and why did the descriptive-normative 
distinction emerge? What assumptions and arguments 
make it possible? How is it related to accounts of ratio- 
nality? What problems has it faced, and what debates 
has it led to? 

One can distinguish between three main phases here. 
I start, in section 2, by showing that from antiquity 
through early modernity, despite important contribu- 
tions to theories of rationality, the descriptive-normative 
distinction existed at best implicitly and thus never 
became a topic of discussion. Next, section 3 argues that 
it was only through a mixture of new metaphysical and 
scientific developments, and importantly a struggle over 
the nature of rationality, that the descriptive-normative 
distinction became prominent, refined, and problema- 
tized. This started in moral philosophy, through David 
Hume’s famous is—ought divide. This, however, is fraught 
with problems. In Immanuel Kant, we find more sophis- 
ticated uses of the is-ought distinction, partly fed by the 
beginnings of empirical research into thinking and deci- 
sion making. Finally, section 4 is devoted to how the 
distinction became sharpened but also contested in the 
late 19th century, within the psychologism debate, pav- 
ing the way for current attitudes toward it. 


2. Antiquity: An Implicit Distinction 
Ancient Greek philosophy has been described as the 


arduous attempt of moving from mythical to rational 
accounts of the world and our place in it (Buxton, 2002; 
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Guthrie, 1952). Indeed, the writings of ancient philoso- 
phers are replete with discussions that influenced sub- 
sequent debates concerning rationality (Frede & Striker, 
1996). Considering the most influential of those authors, 
Plato and Aristotle, will provide a basis for later ideas. 
It will also show the way ancient thinkers argued over 
how far rationality can control other faculties but not 
over whether facts and norms of reasoning are related 
or not. 


2.1 Plato: Logos, the Theory of Forms, and the Good 
In Plato’s dialogues, several expressions, such as logos, 
nous, or dianoia, are used when discussing aspects of 
rationality, in varying and sometimes confusing ways.” 
However, some basic points are uncontroversial. Pla- 
to’s teacher Socrates emerges as a defender of rational- 
ity against the senses, passions, and mere obedience to 
authority. This notion of rationality does not, therefore, 
merely exclude religious or mythical accounts, although 
it does that, too. When speaking of the logos, nous, and 
so on, Socrates and Plato refer to faculties of thinking: 
something over and above perceptions, passions, or 
feelings; something we do to reflect on or take a critical 
stance toward such mental states or dispositions. This 
idea has roots in common ways of talking and acting we 
encounter throughout the ancient world, as seen already 
in Homer’s work. With the advent of philosophy, the 
relevant notions became technical terms and part of 
theories with substantive and controversial assumptions 
(Frede, 1996b). 

For Plato, rationality is not merely a faculty of the 
mind. He thinks that we can exercise rationality because 
we take part in a cosmological order that is itself governed 
by a Jogos. That order guarantees that someone who rea- 
sons well will be right. But how is such taking part in 
the order of the cosmos possible? Plato’s answer is, by 
means of “forms” or “ideas” (ideai or eide): abstract enti- 
ties that make objects what they are.* While not all dia- 
logues contain this same doctrine of “Platonism” or of 
Platonic forms (Patzig, 1970/1996b; Ryle, 1966), some 
(Phaidon, the Republic, or Theaitetos) defend the influ- 
ential claim that perception (aisthesis) is insufficient to 
guarantee knowledge (episteme) of true reality, as it only 
provides access to particular passing appearances (e.g., to 
sensations of color, tastes, or smells). Forms, in contrast, 
can be grasped by the Jogos, now understood as a mental 
faculty, and they capture invariant and general features 
of reality. Among other things, forms enable us to gain 
mathematical knowledge—for Plato, as for many others, 
a paradigm of nonempirical knowledge. 
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Plato’s account of reason can be further explained by 
reference to his doctrine of three powers of the soul: first, 
a rational part (to logistokon, serving to find the true and 
the good and designed to rule the other powers of the 
soul); second, a power of will or courage (to thymoeides); 
third and finally, an appetitive or desiring part (to epi- 
thumetikon, directed toward procuring nourishment and 
sex) (Republic 1V.435b-445b). When the rational power 
rules the soul, the person flourishes, enjoying a healthy 
soul and true happiness. In the early dialogue Protagoras, 
Socrates even argues that having rational insight into the 
good and the right guarantees that one will act accordingly 
and that therefore weakness of the will (akrasia) is impos- 
sible (although in later texts, this position shifts). There is, 
then, no sharp division between theoretical and practical 
rationality; insight into the true and insight into the good 
are intimately entwined with one another. 


2.2 Aristotle: Logic and Science 

Aristotle famously criticizes Plato’s doctrine of forms, 
assigning perception or experience a stronger role in 
his epistemology and also separating theoretical from 
practical rationality more clearly (akrasia, for instance, 
is a definite possibility for him). More important here, 
he understands nous as a power of intuitive insight into 
basic truths of logic, mathematics, and reality, whereas 
logos is a capacity for judgment and for combining 
judgments in arguments or syllogisms. Without these 
insights gained via nous, our reasoning would never 
come to an end and have no foundation. 

One of Aristotle’s most lasting contributions to West- 
ern philosophy is his development of (syllogistic) logic 
(Lear, 1980; Patzig, 1968). He delivers not only a sys- 
tematic account of deduction—the study of correctly 
inferring conclusions from premises on the basis of 
purely formal features—but also already a modal logic— 
possibilities, necessities, and degrees. Furthermore, in 
his On Sophistical Refutations, Aristotle develops a taxon- 
omy of “disputed” (eristikos) or sophistic arguments that 
do not really justify the conclusions they claim to. For 
example, the fallacy of “affirming the consequent” con- 
tains a genuine violation of deductive logic. As Aristotle 
explains, using an example, “since after rain the ground 
is wet in consequence, we suppose that if the ground is 
wet, it has been raining; whereas that does not neces- 
sarily follow” (De Sophisticis Elenchis, chapter 5). He is 
fully aware that people easily produce flawed arguments, 
but he does not enter into any deeper empirical explana- 
tions of why this is so or how facts about human reason- 
ing could be related to its normative rules. 
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Aristotle’s views about nous and logos, together with 
his theory of syllogism, furthermore shape his influen- 
tial account of science (episteme). Deductive validity is 
not enough for science; what is also needed is reasoning 
that amounts to demonstration (apodeixis; see Analytica 
Posteriora). This requires a rigorous regime of reasoning. 
For instance, one needs to structure knowledge in such 
a way that arguments are reordered to begin from initial 
premises that are “better known by nature” (gnorimoteron 
phusei; e.g., APo 71b33-72a25; cf. EN 1095b2-4) or that 
are necessarily true and known to be so. This does not 
mean that all initial premises must be known intuitively, 
or through nous, although ultimately such a state of 
intellectual insight is preferred by Aristotle (APo 99b20- 
100b17; Barnes, 1994). Moreover, scientific demonstra- 
tions not only report facts but also explain them and, 
again, explain them in a way that organizes the plurality 
of explanations so that we start from those that are best 
known. Science is thereby given an axiomatic structure. 
The ideal of scientific rationality does not mean for Aris- 
totle that all good reasoning must proceed from neces- 
sary principles; in many areas, we do not (yet) have such 
starting points. We must start with widely shared opin- 
ions (endoxa), which, if not shared by everyone, must at 
least be shared by the majority or by experts (again, at 
least in their majority) (Top. 100b21-23). From endoxa, 
we then reason our way toward beliefs about controver- 
sial matters by means of dialectical arguments (which 
must also respect the doctrines of syllogistic logic). 

One problem for Aristotle’s views might arise from 
his claim that observation is a genuine source of 
knowledge—a point vivid in De partibus animalium or 
Historia animalium, in which he claims that system- 
atized observations prepare even knowledge of axioms. 
However, while he affirms this, he also emphasizes 
that true knowledge or science essentially depends on 
his strong notion of rationality (Frede, 1996a). Another 
issue concerns whether we can say that Aristotle under- 
stands his ideal of scientific rationality as a model for 
the representation of scientific knowledge and its justi- 
fication, instead of how to proceed in inquiry (Barnes, 
1975). Not all followers of Aristotle agree here, and it 
might be anachronistic to read him this way. In any 
case, his ideal had a lasting influence, shaping the views 
of Descartes, Leibniz, the Logique de Port-Royal (Arnauld 
& Nicole, 1662/1992), Thomas Hobbes, and much later 
authors like Bernhard Bolzano (cf. De Jong & Betti, 
2010). This is so even though it was increasingly chal- 
lenged, especially through recognition of the fallibility 
of theories that had stood for millennia. 
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2.3 Why Was the Descriptive-Normative Distinction 
Not Debated? 

Ancient authors thus developed determinate, as well 
as quite divergent, views on what rationality is in the- 
ory and practice. In addition, authors adopted norma- 
tive viewpoints, partly in response to the possibility of 
error. Plato, Aristotle, and others are clear that failures 
of rationality do occur, even frequently (e.g., in Plato’s 
Republic 438d—439e). In his Nicomachean Ethics, Aristotle 
talks of right reason when he declares that to “act accord- 
ing to the orthos logos is a commonplace, and should be 
assumed” (EN 1103b31-2). Still, while flaws of human 
reasoning and their sources (such as passions or sensory 
illusions) were noted, this did not lead to a discussion of 
how reasoning proceeds as a matter of fact compared to 
how it ought to proceed. Why was there no explicit dis- 
cussion of the distinction and relation between descrip- 
tive and normative perspectives on rationality? 

No literature addresses this question, but we can imag- 
ine possible explanations. One might point to the widely 
held assumption that human rationality participates in 
a cosmological order governed by the Jogos—understood 
now as a principle of the world’s rational structure. 
Another explanation relates to influential assumptions 
of Aristotelian metaphysics: the nature of all beings is 
determined by their form; forms are also always final 
causes. In the case of living beings, the form thus deter- 
mines what the animal is supposed to become and do 
under normal circumstances. If we observe dogs for long 
enough, we discover that they bark, as they should, or 
else they are not truly dogs. If we observe humans for 
long enough, we realize that, qua rational animals, they 
will usually reason as they ought to, despite occasional 
mistakes, or else their rational nature is in doubt. Thus, 
ancient philosophy possessed bridges between “is” and 
“ought.” Normativity seemed to be rooted in reality 
or nature: a foundation safe enough for thinkers from 
antiquity to the Middle Ages. Even though views on 
rationality or reason evolved,’ novel assumptions had to 
emerge for the descriptive-normative divide to become 
fully explicit and a topic worthy of serious debate. 


3. Early Modernity: Toward an Explicit Distinction 
and Debate 


Fast forward to early modernity. Few developments had a 
greater impact on the history of philosophy and science 
than the so-called Scientific Revolution (despite being a 
contested concept) (Henry, 2008; Wootton, 2015). After 
murmurings in the late Middle Ages, its course spanned 
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the work of Copernicus, Kepler, Galileo, Newton, and 
numerous others. It overthrew Ptolemaic cosmology, 
Aristotelian physics, Galenic medicine, and other long- 
accepted belief systems. It changed metaphysical assump- 
tions, and it invited renewed debates about what counts 
as legitimate knowledge and what methods can rationally 
support it. 

Three points matter most for the subsequent history of 
the is-ought distinction. First, developments in philoso- 
phy and science undermined the idea that our rational- 
ity somehow participates in a cosmological order guided 
by the Jogos or by God. Reason or rationality increasingly 
became understood as merely a faculty of individual, 
typically human, minds. In addition, the “new science” 
undermined the Aristotelian view of final causes as a 
metaphysical bridge between “is” and “ought.” Scientific 
explanations were only to be given in terms of efficient 
causality. Second, the Scientific Revolution expanded 
into new domains, such as anatomy, chemistry, econom- 
ics, or psychology. We will see examples of this in specific 
contexts. Third, and most important, rational criticism 
was applied to ever broader domains. The Scientific Rev- 
olution prepared the way for the Enlightenment of the 
18th century: the movement that, in Kant’s expression, 
encouraged human beings to “think for themselves” by 
using the “freedom to make public use of one’s reason 
in all matters,” in order to overcome their “self-incurred 
minority” (Kant, 1900ff., volume VIII, pp. 35, 37). Ancient 
philosophy already contained the germ of such a critical 
function of reason, but it had not spread widely. Now, 
Enlightenment ideas became known all over Europe and 
beyond. In particular, criticism became applied to reason 
itself. The is-ought distinction first became fully recog- 
nized within a lively dispute over reason’s nature, poten- 
tial, and limits, in both theory and practice; then it was 
refined, revised, or criticized. This required several ardu- 
ous steps in philosophical reflection and debate; consider 
René Descartes, David Hume, and Immanuel Kant. 


3.1 Descartes: Rationalism without a Theory 

of Rationality 

Descartes, the founder of modern rationalism (Hatfield, 
2008/2018; Williams, 1978), offers a radical reaction to 
the Scientific Revolution. He is highly optimistic about 
reason’s powers and opens the Discourse on Method 
(1637), if somewhat jokingly, describing “good sense” 
(bon sens) or “reason” (raison) thus: 


Good sense is of all things in the world the most equally 
distributed, for each thinks himself so abundantly pro- 
vided with it, that even those most difficult to please in all 
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other matters do not commonly desire more of it than they 
already possess. It is unlikely that this is an error on their 
part; it seems rather to be evidence that the power of form- 
ing a good judgment and of distinguishing the true from the 
false, which is properly what we call Good sense or Reason, 
is by nature equal in all men. Hence too it will show that 
the diversity of our opinions does not come from some men 
being more rational than others, but solely from the fact 
that our thoughts pass through diverse channels and the 
same objects are not considered by all. (Descartes, 1964ff., 
volume VI, pp. 1-2)° 


This equally distributed faculty of reason is the main 
tool for Descartes’s most influential project: Meditatio- 
nes de prima philosophiae (1641). Here, he proposes that 
every serious thinker, once in a lifetime, can and should 
think through the possibility that all beliefs about reality 
are completely false. If beliefs about, say, the positions 
of the Earth and the Sun, the motions of wandering and 
fixed stars, or the whole Ptolemaic system are based on 
but a gigantic perceptual illusion, why think that any 
other beliefs handed to us by tradition are better off? 
Therefore, Descartes argues, one ought to start from 
scratch if one wants to “establish any firm and perma- 
nent structure in the sciences.” He views it as a demand 
of “reason...to withhold my assent from matters 
which are not entirely certain and indubitable” (Des- 
cartes, 1964ff., volume VII, p. 18). Accordingly, his first 
arguments in the Meditations involve a radical thought 
experiment: there might be a demon who deceives us 
about everything we believe—particularly our empirical 
beliefs about reality. This skepticism needs to be coun- 
tered before one can take controlled steps that erect a 
new foundation of knowledge. 

In this, Descartes builds upon his earlier views about 
what rules one should follow in the exercise of one’s own 
reason. One ought to proceed in a clear order by first 
breaking down all problems into their most simple ver- 
sions, for which one can find simple and clear answers. 
His view on such answers employs the Aristotelian dis- 
tinction between nous and logos in a Latinized form: 
intuitio and demonstratio are two main sources of rational 
knowledge; the former immediate, the latter derivative. 
In the early Regulae ad directionem ingenii, Descartes writes 
that “intuition is the undoubting conception of a clear 
and attentive mind, and springs from the light of rea- 
son (rationis luce) alone” (Descartes, 1964ff., volume X, 
p. 368). In his later work, “clear and distinct perception” 
adopts the role of intuitio, but the core idea remains the 
same: we can know some things with absolute certainty 
because they represent simple, directly evident starting 
points for all inquiry. Ideally, all other knowledge can be 
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derived from them via chains of highly controlled steps 
of reasoning (Descartes, 1964ff., volume VI, p. 19). 

The famous Cogito and Sum offered in Meditation I 
(cf. Perler, 1998, pp. 139-149; Williams, 1978, chapter 3) 
are instances of such basic certainties: our conviction in 
them “is so strong that we have no reason to doubt” (Des- 
cartes, 1964ff., volume VII, pp. 144-145). Descartes adds 
to these certainties the claim that we are equally certain 
about the contents of our mental states. Even when the 
demon might deceive me that there is a light before me, I 
can still be certain that I believe that I see such a light. The 
same with all contents of our beliefs: we might be mis- 
taken that they are true, but we cannot be mistaken that 
we hold them when we do. There are, then, three basic 
Cartesian certainties: I can be absolutely certain that I 
exist, that I think, and what I think. No more is achieved 
at that stage. When Descartes wonders, “What am I?” 
looking for further information about his thinking self, 
and considers the traditional idea of the human being as 
animal rationale, he declares, 


Shall I say a rational animal? No; for then I should have 
to inquire what an animal is, and what is rational; and 
thus from a single question I should insensibly fall into 
an infinitude of others more difficult. (Descartes, 1964ff., 
volume VII, p. 25) 


The rationalist Descartes does not begin from a theory 
of rationality. One consequence of this is that he is not 
clear about the status of basic certainties: they surely can- 
not be a result of voluntary decisions (Newman, 2007). 
Are they, then, something one is psychologically compelled 
to accept? Am I compelled to believe that J think when- 
ever I consider the possibility of an evil demon deceiving 
me, or am I justified in being certain that I think whenever 
I entertain radical doubt? Descartes does not consider this 
vital question. However, the former option would seem 
too weak, as he needs real certainty. But what could guar- 
antee that? He points out that overcoming radical doubt 
requires that one has entertained all possible reasons 
against a proposition. But in order to be more than the 
consideration of doubts a thinker happens to entertain, 
a systematic account of rationality would be required. As 
we have seen, Descartes believes that this would be too 
problematic a starting point. Instead, in Meditation III, he 
famously sets out that these certainties ultimately depend 
on the existence of a benevolent God, and he therefore 
presents a proof of God’s existence and perfection. 

Thus, paradoxically, Descartes’s rationalism is not 
founded on a theory of rationality but on philosophi- 
cal theology. This is a burden that scholars agree his 
arguments cannot bear (e.g., Perler, 1998, pp. 187-202; 
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Williams, 1978, chapter 5). While Descartes defends the 
priority and authority of reason over the senses with 
arguments that must be taken seriously (Frankfurt, 1970; 
Loeb, 1990), and while he allows for a role of the senses 
in science, once reason’s authority is granted, he offers 
no satisfactory rational basis for our beliefs. He is not 
even clear about the status of our most deeply held 
convictions. He simply assumes that there is a “natural 
light” (lumen naturale) of reason that we can rely on: that 
all humans are endowed equally with the power of rea- 
soning and that we can and should “apply it well.” His 
approach lacks a systematic and critical account of the 
nature, foundations, and limits of reason. 


3.2 Hume: Practical Reason 

and the Is—Ought Distinction 

We can take a step forward by turning to Hume. His 
system of empiricism is meant to achieve two things 
at once: to undermine rationalist metaphysics and to 
promote the idea of a thoroughly empirical “science of 
human nature”—hence his celebrated title, A Treatise of 
Human Nature: Being an Attempt to Introduce the Experi- 
mental Method of Reasoning Into Moral Subjects (1739- 
1740/1978; cf. Garrett, 2015; Stroud, 1977). Hume feels 
obliged to consider the nature and limits of reason in 
two main respects: What can reason deliver concerning 
knowledge? How can it guide our actions? His answer to 
the second question presupposes that to the first but not 
the other way around. Both answers include a severe 
restriction of the powers of reason, and they ultimately 
lead him toward his is—-ought distinction. 

Concerning its relation to knowledge, Hume claims 
that reason is “the discovery of truth and falsehood.” 
More specifically, reason can discover knowledge of either 
“matters of fact” or “relations of ideas” (Hume, 1739- 
1740/1978, pp. 448, 458): empirical knowledge (which 
is contingent and uncertain) or a priori knowledge (the 
contradictory opposite of which is impossible). The lat- 
ter can—using the intuitio-demonstratio distinction—be 
either intuitively or demonstratively certain. Hume also 
assumes that all we can know empirically must be based 
on sensory “impressions,” and thus, all our claims about 
the future or past, and about what causes impressions, 
have at best the status of probable belief. Most impor- 
tant, he rejects the idea that reason can deliver metaphysi- 
cal knowledge (e.g., of God, or of a difference between 
mind and body). To demonstrate this, he develops skep- 
tical arguments concerning the possibility of such alleg- 
edly rational knowledge in three areas: causal or inductive 
inference, the existence of material bodies, and personal 
identity. Nonetheless, Hume thinks that human nature 
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means we must hold beliefs on these topics. This is not 
a rational “must” or “ought”; it is simply something we 
unavoidably do in our normal lives. Thus, although there 
is no demonstrative proof of the “principle of induction,” 
we cannot but form inductive beliefs based on habits and 
experiences (Winters, 1979, pp. 26-29). 

As to action, two points are paramount. On the one 
hand, against the tradition going back to Socrates, Hume 
rejects the idea that we have rational control over our pas- 
sions or desires. He famously declares, “Reason is, and ought 
only to be the slave of the passions, and can never pretend 
to any other office than to serve and obey them” (Hume, 
1739-1740/1978, p. 415) and offers colorful examples: 


Tis not contrary to reason to prefer the destruction of the 
whole world to the scratching of my finger. ’Tis not con- 
trary to reason for me to chuse my total ruin, to prevent the 
least uneasiness of an Indian or person wholly unknown to 
me. ‘Tis as little contrary to reason to prefer even my own 
acknowledg’d lesser good to my greater, and have a more 
ardent affection for the former than for the latter. (Hume, 
1739-1740/1978, p. 416) 


Some important clarifications are needed here. First, 
although reason is entirely inactive concerning passions, it 
can still help to direct our actions. Based on our best beliefs 
concerning cause-effect relations, we can determine ratio- 
nal ways to realize our passions (Hume, 1739-1740/1978, 
p. 459). This is called instrumentalism about practical rea- 
son (e.g., Audi, 2001, p. 5; Korsgaard, 1996, p. 312), but 
this instrumentalism involves reason as only a passive, 
cognitive faculty. It therefore cannot provide a foundation 
of morality. It is precisely at this stage of his thought that 
Hume introduces his famous is—ought distinction: 


In every system of morality, which I have hitherto met 
with ...the author proceeds for some time in the ordi- 
nary way of reasoning, and establishes the being of a God, 
or makes observations concerning human affairs; when of 
a sudden I am surprised to find, that instead of the usual 
copulations of propositions, is, and is not, I meet with no 
proposition that is not connected with an ought, or an ought 
not. This change is imperceptible; but is, however, of the last 
consequence. For as this ought, or ought not, expresses some 
new relation or affirmation, ‘tis necessary that it should be 
observed and explained; and at the same time that a rea- 
son should be given, for what seems altogether inconceiv- 
able, how this new relation can be a deduction from others, 
which are entirely different from it. But as authors do not 
commonly use this precaution, I shall presume to recom- 
mend it to the readers; and am persuaded, that this small 
attention would subvert all the vulgar systems of morality, 
and let us see, that the distinction of vice and virtue is not 
founded merely on the relations of objects, nor is perceived 
by reason. (Hume, 1739-1740/1978, p. 469) 
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Hume thinks that the is-ought distinction—“Hume’s 
Guillotine” (Black, 1964)—subverts theories of ethical 
rationalism such as those of Samuel Clarke (1675-1729) 
or (perhaps) John Balguy (1686-1748),° which start from 
factual assumptions about human conduct or—more 
subtly—about how actions are related to one another 
in an imagined rational order (created by God). For 
instance, giving a gift is “in agreement with” gratitude, 
or committing a crime is “in agreement with” punish- 
ment (Balguy, 1729, pp. 7-8). Balguy thought that reason 
affords, through comparing ideas of actions, (partly self- 
evident) knowledge of what actions agree or disagree with 
one another and hence what our obligations are. Hume 
claims that such an inference constitutes a fallacy and 
that, once this is recognized, rationalistic moral theories 
are left hanging in the air. For Hume, reason is a purely 
cognitive faculty: all it provides us with is knowledge of 
either empirical “matters of fact” or analytical “relations 
of ideas.” Neither option provides knowledge of how 
humans ought to act. The real basis of “ought” statements 
must therefore be sought outside reason: in the passions. 

Hume’s views have attracted many followers. Ber- 
trand Russell accepted instrumentalism about reason— 
albeit not in terms of “passions” but of “ends”: 


‘Reason’ has a perfectly clear and precise meaning. It signi- 
fies the choice of the right means to an end that you wish 
to achieve. It has nothing whatever to do with the choice of 
ends. (Russell, 1954, p. viii) 


More recently, Robert Nozick has noted that instru- 
mentalism is “powerful and natural”: indeed, the “default 
theory” of practical rationality (Nozick, 1993, p. 133). 

In its explicitness, Hume’s is-ought distinction consti- 
tutes a historical novelty. It provoked critical reactions, 
for instance, from Adam Ferguson (1769) or Thomas 
Reid (1788/1827) and other, more positive reactions, 
such as those of 20th-century ethical emotivists (e.g., 
Stevenson, 1944). Hume paved the way for discussions 
on the relation between facts and norms, descriptions 
and prescriptions, and similar issues. However, neither 
his instrumentalism nor the is-ought distinction are 
unproblematic. I now present two problems concerning 
each of these two ideas (cf. Hepfer, 1997, pp. 115-126; 
Setiya, 2004; Stroud, 1977, pp. 154-170). 

First, even though the is—ought distinction appears to 
express a Clear, logically valid point, there may be coun- 
terexamples to it. Consider the following: 


1. Beer drinking is common in Belgium. 


Therefore, beer drinking is common in Belgium or all 
banks ought to be socialized. 


Theories of Rationality and the Descriptive-Normative Divide 


Such an argument seems to refute the universal valid- 
ity of Hume’s Guillotine. Debates about such examples 
continue (cf. Hudson, 1969; Pigden, 2010; Schurz, 1997, 
2010; Singer, 2015; Wolf, 2015), but Hume’s view may be 
an overgeneralization. However, this misses a core point of 
his argument. Not only do such counterexamples use tricks 
(such as extension by disjunction), but it is also doubt- 
ful that Hume wanted to establish a completely general 
is-ought distinction. He has a distinct target: rationalistic 
ethical theories that derive moral norms from facts about 
behavior or from alleged necessary relations between ideas 
of actions. Hume gives an account of what reason can and 
cannot do, and on the basis of that account, he views the 
inference to “oughts” as unwarranted. 

The second problem concerns Hume’s assumption 
that any “ought” is derived from passions. Even if one 
grants that reason cannot determine or control passions, 
one may doubt that these can justify any “ought” state- 
ments. To say that passions have strong causal force 
and that we feel pushed by them to carry out certain 
actions does not imply that we therefore ought to per- 
form those actions. Conversely, even if there are instru- 
mental reasons to perform an action (e.g., we ought to 
avoid unhealthy food if we want to stay healthy), it does 
not follow that we will be motivated to do so. If Hume 
throws the is—ought distinction at rationalists, they can 
throw it back at him just as well. 

A related point comes from Thomas Reid. He attacks 
Hume by claiming that ethical theories do not start from 
factual premises but from normative ones: from basic, 
“self-evident” insights into our duties from which nor- 
mative conclusions can be deduced (Reid, 1788/1827, 
p. 661). If one accepts this strategy, the fallacy is circum- 
vented. However, this requires a convincing account of 
the normative basis of reason. Otherwise, one might end 
in skepticism concerning the normativity of practical 
reason in general—not only noninstrumental or moral 
but instrumental reason as well. There is suggestion that 
Hume is forced to adopt such skepticism (Hampton, 
1995; Korsgaard, 1996, pp. 312-314; Setiya, 2004). 

Third, one might claim that reason functions not 
only to steer actions toward the realization of goals but 
also to systematize our plurality of goals. An assessment 
and hierarchical ordering of ends serves the long-term or 
enlightened self-interest of agents and helps to exclude 
certain forms of irrationality. For these reasons, one 
should incorporate this requirement (as current theories 
of rational choice do). Hume, however, as we have seen, 
maintains that it is not “contrary to reason for me to 
chuse my total ruin” and “as little contrary to reason 
to prefer even my own acknowledg’d lesser good to my 
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greater” (Hume, 1739-1740/1978, p. 416). This follows 
from his assumptions that reason only functions cogni- 
tively and that passions never represent objects and so 
cannot be true or false (Hume, 1739-1740/1978, pp. 415, 
458). This might be questioned as well: even though pas- 
sions cannot be true or false, they have an “intentional” 
character—they concern what states of affairs we intend 
to bring into existence. Such intentions, at least many of 
them, might be evaluable by reason too (Stroud, 1977, 
pp. 162ff.). 

Finally, is instrumentalism about reason a descriptive or 
a normative claim? The problematic answer is: it is both. 
In line with his “science of human nature,” Hume aims 
to discover an empirically correct conception of the rela- 
tion between reason and passion (Hampton, 1995; Win- 
ters, 1979). One might push this further by saying that 
his goal is not to justify “oughts” but merely to explain 
them. That is, Hume would only be interested in telling 
us why people speak in normative terms in their moral 
conduct, without endorsing the validity of such talk. 
However, his writings are replete with normativity. He 
introduces his instrumentalism as a claim that reason “is, 
and ought to be” the slave of the passions. Given Hume’s 
Guillotine, the phrase “is, and ought to be” can hardly be 
a slip of the tongue. Furthermore, his instrumentalism is 
directed against rationalistic ethical theories that have, of 
course, normative pretensions: “Every rational creature, 
it is said, is obliged to regulate his actions by reason; and if 
any other motive or principle challenge the direction of 
his conduct, he ought to oppose it, till it be entirely sub- 
dued, or at least be brought into conformity with that 
superior principle” (Hume, 1739-1740/1978, p. 413, ital- 
ics added). If Hume’s instrumentalism regarding practical 
reason is merely a descriptive thesis, then ethical ratio- 
nalists can reject it as irrelevant—and perhaps false, by 
claiming that we can (sometimes) control our passions, 
even if it is difficult. The arguments that Hume devel- 
ops against such a view must express more than just 
empirical truths about reason and passion. Indeed, his 
considerations seem more fundamental. Hume’s Guillo- 
tine appeals to a basic logical distinction. Similarly, his 
assumption that passions are never representations of 
objects, and so cannot be true or false, does not look like 
an empirical claim either. Hume wants to be a scientist 
of human nature. While this leads him to grapple with 
the problem of how descriptive and normative claims 
concerning morality are related, he adopts no clear and 
convincing stance on it. Perhaps this is because he intro- 
duces the is—ought distinction in order to attack versions 
of ethical rationalism. Hume lays the ground for the dis- 
tinction between the descriptive and the normative, but 
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he does not consider its extent or think through its impli- 
cations. Most important, he has no convincing account 
of the sources of normative rationality. 


3.3. Kant: Applying the Is-Ought Distinction 

in Philosophy and Science 

The “all-crushing Kant” (Moses Mendelssohn) delivers 
an even more devastating criticism of rationalistic meta- 
physics, as represented especially by Leibniz, Christian 
Wolff (1679-1754), and their followers. Kant radically 
reconceptualizes reason, restricting its theoretical func- 
tions and expanding its role in practical philosophy (see 
Guyer, 2006). In doing so, he understands and uses the 
is-ought distinction in more sophisticated ways than 
Hume. Although the debate over the foundations of 
morality remains important (Guyer, 2016), Kant’s use of 
the distinction also goes beyond that domain. He repeat- 
edly claims that while certain disciplines or sciences deal 
with facts of human thought, desire, and action, others 
are concerned with irreducibly normative questions 
concerning such mental activities, and he thoroughly 
addresses the sources of normativity. 

In his opus magnum, the Critique of Pure Reason (1781, 
2nd rev. ed. 1787),’ Kant argues that reason cannot arrive 
at knowledge claims about reality independently of expe- 
rience. Attempts to prove metaphysical claims (about 
an immaterial soul, the existence of God, etc.) lead to 
“transcendent illusions.” Pure reason, through its infinite 
quest for explanations and justifications, leads us to fall 
prey to these illusions (Kant, 1781/71787/1998, A vii). 
We cannot avoid them, just as we cannot avoid percep- 
tual illusions (Kant, 1781/71787/1998, A295/B351-352, 
A297/B354), even when we grasp their illusory nature.® 
Therefore, metaphysics is an inescapable consequence 
of human reason (Willaschek, 2018). However, we can 
learn to see through illusions of reason by scrutinizing 
the reasoning patterns that lead to them. It is this project 
that leads Kant to think that reason is not only there to 
criticize existing systems of belief and conduct but that in 
order to do so properly, it must critically examine itself. 

Such self-examination, to pick out the central construc- 
tive doctrine of the Critique, leads Kant to determine the 
boundaries of all possible knowledge claims by means of 
a system of synthetic a priori principles: principles (such 
as the principle of causality) that are necessary for making 
“objective” knowledge claims (i.e., claims that are deter- 
minately true or false about real objects). Synthetic a priori 
principles are not a product of reason but of cooperation 
between “pure understanding” (reiner Verstand) with its 
concepts, the so-called categories, and “pure sensibility” 
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(reine Anschauung) with its a priori forms of space and 
time. Kant describes understanding, alongside the facul- 
ties of judgment (Urteilskraft) and reason, as the “higher” 
cognitive faculties (and sometimes speaks of reason in 
confusing ways where he means understanding’). Tradi- 
tional rationalist metaphysics violates the limits of possi- 
ble knowledge determined by synthetic a priori principles. 

Having refuted traditional rationalist metaphysics, 
Kant thinks that he needs to show that reason has wholly 
different tasks. This can be connected to the is—ought 
distinction. While he agrees with Hume that the dis- 
tinction between “is” (Sein) and “ought” (Sollen) is deep, 
fundamental, and in principle easy to recognize, Kant 
connects it with quite different accounts of reason. First 
and foremost, he uses the is-ought distinction to sepa- 
rate theoretical from practical philosophy: the former 
concerns only nature, the realm of things as they are in 
fact, whereas the latter concerns only how things ought 
to be (Kant, 1781/71787/1998, A319/B375, A633/B661). 
This is a distinction between the main domains of phi- 
losophy, and Kant also states that reason is a “lawgiver” 
for each of them: laws of nature for theoretical and laws 
of morality for practical philosophy (A840/B868). Second, 
and cutting across this basic divide, Kant applies the is— 
ought distinction at the level of our making judgments, 
or even of pursuing systematic or scientific investigations 
within these two domains. Thus, in theoretical philoso- 
phy, although its claims concern Sein, we can nonetheless 
make normative claims too and actually have to: theo- 
retical disciplines such as logic provide norms for good 
reasoning about facts. Here, Kant applies the is-ought 
distinction via warning that one cannot derive the laws 
of logic from psychological laws of actual human think- 
ing. It is a frequent Kantian claim that when dealing with 
recognizably different tasks, one ought to treat them in 
different disciplines: we “do not enlarge but disfigure sci- 
ences, if we allow them to trespass upon one another’s 
territory” (Kant, 1781/21787/1998, B viii). 

Kant’s sophisticated use of the is-ought distinction 
has advantages over Hume’s. Given Hume’s project of an 
empirical “science of human nature,” perhaps he wishes 
only to explain why people make demands or state obli- 
gations, without considering the normative validity of 
such talk. But Hume’s instrumentalist account of prac- 
tical reason certainly looks normative, and as argued 
above, he makes many other normative claims too. In 
contrast, Kant recognizes that we make normative judg- 
ments and that such discourse can then be studied either 
descriptively (and empirically) or normatively (ethically). 
This provides greater clarity. 


Theories of Rationality and the Descriptive-Normative Divide 


Now, in moral philosophy, Kant’s specific concept of 
“pure practical reason” plays a strong, constitutive role. 
While Hume uses the is—ought distinction to fight ethi- 
cal rationalism, Kant applies it to defend his own ratio- 
nal ethics. For Hume, practical reason is at best applied 
theoretical reason; for Kant, it governs a territory where 
it sets up its own, irreducibly normative, moral laws of 
conduct. Reason, he argues in Groundwork to the Metaphys- 
ics of Morals (1785), helps to identify and justify the basic 
principle of morality: the “categorical imperative.”'° We 
understand moral rules as universal and necessary. First, if 
a moral norm is valid, then it is valid for all agents in 
similar situations: making an exception for oneself is 
immoral. The fact that we often observe people deviating 
from norms does not invalidate them. Second, a moral 
rule does not describe, but demands, certain courses of 
conduct; this “ought” character constitutes its necessity. 
Kant combines these claims with different disciplines 
studying rational action. Thus, “anthropology” studies 
human actions empirically; ethics or the “metaphysics 
of morals,” in contrast, considers how we ought to act 
(Kant, 1900ff., volume IV, pp. 389, 411-412; volume VI, 
pp. 216-217). Note that when Kant speaks this way, he 
means only moral normativity. As should be emphasized, 
he also makes room for “merely” instrumental normativ- 
ity, in his terminology, “hypothetical imperatives”; also, 
despite differences in detail, he agrees with Hume that 
such imperatives can be grounded in empirical means-— 
ends knowledge (Kant, 1900ff., volume IV, pp. 413-419). 

How does Kant, again in contrast to Hume, also come 
to use the is—-ought distinction within theoretical philoso- 
phy? As indicated above, the Scientific Revolution led to 
the idea that not only the physical but also the mental 
world might be studied scientifically. Kant is quite famil- 
iar with ideas of a distinct empirical science of human 
thinking, feeling, desiring, and acting (Sturm, 2009, 
chapter 4). Such studies were usually called “empirical 
psychology”: a pet project of Wolff and his disciple Alex- 
ander Baumgarten (1714-1762), but also of critics such 
as Johann Nikolaus Tetens (1736-1807). They refined 
folk-psychological or folk-philosophical theories of the 
architecture of mental faculties. At the same time, some 
psychologists developed the first experimental and quan- 
titative studies of mental processes. Thus, Gottlob Fried- 
rich Hagen (1710-1774) measures the “perfection” of the 
power of reason by the number of extended syllogisms 
one can understand and repeat and by how many dem- 
onstrations one can produce (Hagen, 1733, §§12-14). 
Kant emphasizes that such empirical psychology contains 
factual, contingent statements and explanations (Kant, 
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1900ff., volume XXV, pp. 243, 473). Because of this, it 
would be misguided to use psychology in logic: 


Logic is a science of reason . . . a science a priori of the neces- 
sary laws of thought... hence a science of the correct use 
of the understanding and of reason in general, not subjec- 
tively, however, i.e., not according to empirical (psychologi- 
cal) principles for how the understanding does think, but 
objectively, i.e., according to principles a priori for how it 
ought to think. (Kant, 1900ff., volume IX, p. 16) 


This richer background distinguishes Kant from Hume, 
who is unclear about how far his “science of human 
nature” is descriptive, normative, or both. Still, whether 
the Kantian view of logic is convincing became much dis- 
puted in the next century (cf. section 4). 

Alongside ethics and logic, a further area where Kant 
brings the is—ought distinction into play is in his philos- 
ophy of science. “Reason,” now in the sense of an alleged 
power of metaphysical knowledge, produces its own 
concepts and principles (Kant, 1781/71787/1998, A289- 
290/B355-356). As noted, even when Kant claims that 
we should not view such “ideas of pure reason” as repre- 
senting possible knowledge of real objects, he thinks that 
we cannot avoid having them. Do these ideas have any 
other function, then? The critical examination of reason 
leads Kant to redefine one of reason’s functions: ideas 
of God, the soul, or the world as a whole play a posi- 
tive role in our theoretical knowledge by becoming used 
not “constitutively” (ie., referring to objects) but “regu- 
latively” in scientific research (Kant, 1781/71787/1998, 
A311/B368, A647/B675): they prescribe to us how to sys- 
tematically connect pluralities of empirical judgments 
into systematized bodies of theories. Thus, judgments 
about stellar positions, motions, velocities, and the 
structure of the planetary system become unified in the 
theories of Copernicus, Kepler, Newton, and in Kant’s 
own cosmology (containing his famous nebular hypoth- 
esis), all guided by the notion of the world as a system- 
atic whole governed everywhere by the same laws. The 
normative aspect here is that we ought to aim at such 
systematized bodies of knowledge. Moreover, there are 
certain normative rules, such as Occam’s razor (“Do not 
multiply entities beyond necessity”) or continuity of 
forms (e.g., “Make division of things in chemical, bio- 
logical, or psychological taxonomies as complete as pos- 
sible”), for doing so (Kant, 1900ff., volume V, p. 182). 
We might never arrive at a truly finished theory in any 
domain; in Kant’s view, science is forever an open-ended 
enterprise. However, that we ought to pursue that aim 
is reasonable: it supports scientific progress (cf. Neiman, 
1994, chapter 2; Wartenberg, 1979). 
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What to think of Kant’s claims concerning the is— 
ought distinction? I will briefly consider here two objec- 
tions concerning his ethics. First, he claims that reason 
is not a purely passive, cognitive capacity but “has cau- 
sality, or that at least we can represent something of 
the sort in it” and that this “is clear from the impera- 
tives that we propose . . . in everything practical” (Kant, 
1781/°1787/1998, A547/B575). Hume would reject this 
claim about a “causality” or activity of practical reason, 
and we might wonder on whom the burden of proof 
falls here. Kant’s qualification “at least we can represent 
something of the sort in it” shows he is perhaps not cer- 
tain about the causality either. In his ethical works, he 
develops deeper accounts of “pure practical reason”— 
although it is disputed whether he succeeds with them. 

Second, and related to this, his notion of pure practi- 
cal reason seems too idealistic to provide norms that are 
realistic for human beings. Can we really follow moral 
norms? Is Hume not right that reason is—and ought 
to be—dominated by passions? This popular objection 
might explain why Kant’s notion of pure practical rea- 
son did not become incorporated into modern theories 
of rational choice or economic decisions, which usually 
tried to be ethically neutral (but see Bjorndahl, London, 
& Zollman, 2017; Churchman, 1970). 

Partly, this criticism rests on a limited understand- 
ing of Kant’s practical thought. He does not maintain 
that empirical study of human history or society treats 
thought or action as arational processes. His “pragmatic 
anthropology” aims to study the human being as an 
animal endowed with reason (Kant, 1900ff., volume VII, 
p. 119; Sturm, 2009, chapter 8). In this, Kant accepts that 
human beings can be imperfectly rational: the possession 
of the faculty of reason is one thing; its improvement is 
another. In this vein, he writes that the human being is 
not an animal rationale but an animal rationabile (Kant, 
1900ff., volume VII, p. 321). Anthropology can teach us 
how to improve our rationality—not the basic compe- 
tence that normal human beings all possess but our per- 
formance in following its norms (or developing better 
ones)—in “pragmatic” ways, that is, in ways that in the 
long run might even support the realization of morality. 
This requires a historical and social process. Still, Kant 
insists that one cannot derive categorical moral norms 
from such pragmatic anthropological knowledge. 

To sum up: while Descartes neither has an account of 
rationality that could ground his normative pretensions 
nor states the is—-ought distinction, his attempt to estab- 
lish metaphysical truths provokes Hume as well as Kant 
to develop accounts of reason that enable them to attack 
such metaphysics. Against this background, Hume uses 
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his strictly cognitive notion of reason to argue that ethi- 
cal rationalism fails, among other things, because reason 
cannot establish any moral “oughts.” Kant, who accepts 
the is-ought distinction, develops its uses further by 
applying it both to distinguish the two realms of theo- 
retical and practical reason and to distinguish between 
descriptive and normative investigations within each 
domain. The background motivation for this is his sus- 
tained attempt to understand reason and all its potential 
as well as its limits. While his account can be considered 
an advance on Hume’s, it is not without problems either. 


4. The Psychologism Debate 


Two major developments determined the subsequent 
evolution of the debate over the is-ought divide, primar- 
ily within contexts of theoretical philosophy. To under- 
stand why they matter for us, we must keep in mind 
that rationality has traditionally been connected to the 
theory of logical inference.'' In Kant, this is the most 
basic and uncontroversial notion of reason, and his view 
of logic is unapologetically normative and antipsychol- 
ogistic. Even in the 20th century, much cognitive psy- 
chology concerning reasoning starts from experiments 
involving logical tasks (see Sturm, 2012; chapter 1.2 by 
Evans, this handbook). 

The first development consists in the rise of neo- 
Kantianism in the late 19th century, after a phase of 
dominance of German idealism (Heis, 2018). Philoso- 
phers such as Eduard Zeller (1814-1908) or Hermann 
Cohen (1842-1918) demand that the metaphysics of 
Hegel and his followers be replaced by a logic married to 
Erkenntnistheorie (“epistemology”: a term invented in the 
early 19th century; Vaihinger, 1876). As Zeller declared 
in an influential speech, idealists view logic as “not only 
knowledge of the forms of thought but knowledge of 
reality as well” (Zeller, 1877, p. 480). 

The second development concerns the further advance 
of scientific psychology. In the 19th century, this led to 
the formation of the first academic professorships, insti- 
tutions, and societies of psychology (Gundlach, 2005). 
Much of the work began with perception, but it also 
became wedded to previous empiricist conceptions of 
logic and reasoning. As Zeller (1877) declares, there was 
“an older” notion of logic “concerned with the activ- 
ity of thought as such” (pp. 479-480). Indeed, many 
early modern thinkers saw logic as normative but none- 
theless dependent on actual mental processes and their 
laws (Hatfield, 1997). John Stuart Mill, for whom logic 
is the “science of reasoning” and studies “the operations 
of the human understanding in the pursuit of truth” 
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(Mill, 1843/1973-1974, p. 6), declares that knowledge of 
laws of thinking is based on “the ordinary methods of 
experimental inquiry” (Mill, 1843/1973-1974, p. 853) 
and maintains that “the general laws of association pre- 
vail among these more intricate states of mind” (Mill, 
1843/1973-1974, p. 856). A similar view can be found in 
Wilhelm Wundt (1832-1920), the influential organizer 
of psychology in the late 19th century. 

Against such conceptions, Zeller emphasizes the 
strictly formal character of logic and demands it to be sep- 
arated from both metaphysics and psychology and con- 
nected with epistemology instead. Thus, he declares logic 
to be “the scientific methodology” that is presupposed 
in any “investigation of the real” (Zeller, 1877, p. 481), 
pointing to Kant for this (Zeller, 1877, p. 490). The rise 
of neo-Kantianism and academic psychology thus leads 
to a clash over the status of logic (and, thereafter, of epis- 
temology and philosophy of science). The psychologism 
debate involves numerous contenders; only some major 
positions and arguments can be considered here (Carl, 
1994, chapter 1; Kusch, 1995, 2007/2015). 

For instance, Theodor Lipps (1817-1881) criticizes the 
Mill-Wundt view that logic, while based on empirical 
laws of thinking, is a normative discipline. Lipps describes 
logic “as the physics, not the ethics, of thinking” (Lipps, 
1880, p. 529). In addition, he argues that we must dis- 
tinguish between prescriptivity and normativity: logi- 
cal rules are not the result of a voluntary decision or an 
“authoritative will” and so should not be called prescrip- 
tive. Instead, laws of logic are like “norms of nature,” and 
therefore logic is “the physics of thinking or it is nothing 
at all” (Lipps, 1880, p. 531). Wundt responds that Lipps 
presents a false dichotomy, since logic can be based on 
descriptive laws insofar as “our thinking is not influenced 
by anything which interferes with its being governed by 
these laws” (Wundt, 1882, p. 345). 

One of the most influential antipsychologism 
positions—often characterized as decisive—stems from 
the philosopher-mathematician Gottlob Frege (1848- 
1925). Frege is viewed as the founder of modern logic 
and of analytic philosophy. His main project was to 
show that mathematics or, more specifically, arithmetic 
could be reduced to logical laws. Since the logics as Frege 
found them were insufficient to support such a reduc- 
tion, he created a novel, axiomatized predicate calculus 
with innovative tools for complex quantification. 

Frege’s antipsychologism concerning logic (cf. Carl, 
1994, chapter 1) is based on the claim that the validity of 
logical laws is independent of how people actually think 
and reason (Frege, 1956). As he explains, “We can inquire, 
on the one hand, how we have gradually arrived at a given 
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proposition and, on the other, how we can finally provide 
it with the most secure foundation” (Frege, 1879/1972, 
Preface). This may seem like a merely gradual distinction, 
but it is not. To begin, Frege points out that the “first 
question may have to be answered differently for differ- 
ent persons; the second is more definite, and the answer 
to it is connected with the inner nature of the proposition 
considered” (Frege, 1879/1972, Preface). With the latter 
point, what Frege refers to is what logic should be all 
about. Logical rules determine whether inferences from 
premises to conclusions are truth preserving. Whether 
the premises of an argument are true often cannot be 
decided by logic alone, but logic tells us which inference 
patterns guarantee that if the premises are true, then the 
conclusion will also be. The validity of logical laws has to 
do, among other things, with the meaning of the logical 
connectives” such as “->” (“if-then,” interpreted as the 
material conditional) and “-” (negation). Thus, modus 
ponens (p — q; p; therefore, q) and modus tollens (p— q; 74; 
therefore, =p) are deductively valid under any interpreta- 
tion of the propositional variables p and q. That human 
reasoning sometimes violates and sometimes conforms to 
such valid patterns is, from Frege’s point of view, irrel- 
evant for their logical validity. 

As seen in Mill and Wundt, psychologism can be com- 
bined with the view that logic is normative—something 
Kant would have denied. Conversely, proponents of anti- 
psychologism do not necessarily think of logic as nor- 
mative or as a kind of “ethics of thinking.” Frege writes 
more cautiously that logic is “different from psychology, 
related to ethics” (Frege, 1979, p. 1): he avoids calling 
logic a kind of ethics. He does not view the laws of logic 
as inherently normative but descriptive. Descriptive of 
what? Logical rules purport to be universal and time- 
lessly true. Frege (1956, p. 302) speaks of a “third realm” 
(drittes Reich) beyond the material world and our subjec- 
tive mental representations. This is much disputed (Carl, 
1994, chapter 8), but what matters is the innocent point 
accepted since Aristotle: given that we can make mistakes 
in our reasoning, we ought to correct them, and this can 
involve logical laws. So, these laws have a prescriptive 
function, but in themselves, Frege views them as strictly 
descriptive. 

Frege’s antipsychologism, his distinction between an 
explanation of the genesis of our beliefs and a possible 
justification of their grounds, revived Kantian views. 
While his and others’ (especially Edmund Husserl’s) 
arguments became widely accepted, the status of logic 
remains an open question. Logic after Frege has become 
enriched not only by deontic and modal logics but also 
by more specialized or applied theories such as relevance 
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logic or defeasible reasoning. These address special prob- 
lems of human reasoning where classical logic often fails 
(see, e.g., sections 3, 5, 6, and 7, this handbook). 


5. Conclusion: Toward the 20th Century 


While Frege focused on logic, his views became influen- 
tial beyond this domain, especially through the episte- 
mology and philosophy of science of the Vienna Circle. 
Not all reasoning, after all, is deductive. Many nonde- 
ductive inferences, for instance, about scientific hypoth- 
eses, use probabilities. Logical Empiricists such as Hans 
Reichenbach (1938) or Rudolf Carnap (1950) tried to do 
for probability theory or “inductive logic” what Frege 
had achieved for deductive logic: design an axiomatic 
system with clear tasks, limits, and structure. They spoke 
of epistemology and philosophy of science as “logic of 
science.” However, this requires that all philosophical 
claims about knowledge or science are formal or logical 
truths, which is highly implausible. Still, even critics of 
the Vienna Circle, like Karl Popper, viewed the distinction 
between “is” and “ought” as basic for (not as identical 
to!) both that between “discovery” and “justification” of 
beliefs (Reichenbach, 1938, chapter 1) and that between 
tasks of psychology and epistemology. They thereby tried 
to carve out a territory for philosophy of science and its 
theories of confirmation, falsification, scientific expla- 
nation, and so on (Peckhaus, 2006). With the advent of 
naturalistic and historicist approaches in philosophy of 
science, the is-ought, the descriptive-normative, and the 
discovery-justification distinctions have become ques- 
tioned again (e.g., Giere, 1989; Hoyningen-Huene, 1987; 
Kitcher, 1992; Schickore & Steinle, 2006). 

I have argued that the divide between the descrip- 
tive and the normative has an interesting and instruc- 
tive history. Meanings, together with debates over them, 
often depended on competing assumptions about what 
rationality is, as well as on agendas and assumptions in 
metaphysics, ethics, logic, and the sciences. Bringing this 
history to light should enable us to understand better 
and to reconsider our current situation. The divide often 
poses challenges for current approaches to the study of 
rationality, particularly in interdisciplinary contexts. In 
any case, the demand to reflect critically on the norma- 
tive authority of rationality and its relation to empirical 
studies of human reasoning is alive and well. 
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Notes 


1. I do not here consider the view that reason and rationality 
should be distinguished, in a way that approximately mirrors 
the distinction between instrumental, means-ends reasoning 
and (moral) reasoning concerning ends themselves (e.g., Elster, 
2009; Rawls, 1980; Sibley, 1953). For criticisms of such distinc- 
tions, see Patzig (1994/1996a) and Sturm (2019). In collabora- 
tion with others (Erickson et al., 2013), I have argued that the 
term “rationality” gained popularity during the 20th century, 
tending to push “reason” into the background. “Rationality” 
often became understood as a mechanical or “mindless” appli- 
cation of formal and optimizing rules, exemplified by logic 
and rational choice theory, whereas “reason” referred more to 
a mindful or reflective use of the faculty of thinking. The ter- 
minology has not always followed this trend, however. Here, I 
will speak primarily of “rationality” and resort to “reason” and 
related terms when this is more appropriate for the source texts. 


2. For example, logos has many meanings, such as “speech,” 


you nou you nou 


“explanation,” “ratio,” “computation,” “reckoning,” “argu- 
ment,” and “law,” to mention but a few from the entry in 
Liddell-Scott’s Greek-English Lexicon. Moss (2014) argues that 
logos in Plato’s and Aristotle’s works should often be translated 


as “reason,” related to explanatory accounts. 


3. The masterful analogies of the sun, the divided line, and 
the cave (Republic VI. 507b-511e; VII. 514a—520a) present the 
theory of forms in vivid ways. 


4. Cf. Frede and Striker (1996) and Rapp et al. (2001). In an 
independent philosophical tradition, namely Chinese, we 
find related philosophical problems (e.g., about norms for our 
practical lives and an awareness that these can be violated), 
but again, there is a lack of philosophical reflection on the is— 
ought gap even beyond the Middle Ages (Marchal & Wenzel, 
2017). How far the emergence of a discussion of the is-ought 
gap is an exclusively Western development deserves further 
investigation. 


5. While the references use the standard Adam-Tannery edi- 
tion, translations are from the edition by Haldane and Ross 
(Descartes, 1934ff.), with minor improvements. 


6. It is not certain that Hume knew Balguy’s views. However, 
they provide an illuminating background, since Balguy debated 
the basis of moral obligations with an anonymous critic. The 
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critic’s objections were published together with Balguy’s replies 
(Balguy, 1729). For instance, the critic argued that if we do not 
already have the idea of obligation or “ought,” then the inspec- 
tion of ideas such as “Gratitude, Ingratitude, and Bounty” by 
reason, independently of sentiments, “could never so much 
as afford us a general Idea of Obligation in itself; or inform us 
what is meant by that Term; much less could we be able to 
deduce the particular Obligation to Gratitude from these Ideas” 
(Balguy, 1729, p. 8 [Art. IV of the criticism]; cf. pp. 9-16). This 
is an anticipation of Hume’s is—ought distinction and similar to 
other aspects of his ethical theory. 


7. Citations from Kant’s work refer to the Academy edition 
(Kant, 1900ff.) and the standard edition of the first Critique but 
using the English translations (Kant, 1992ff.). 


8. A similar analogy is drawn in current heuristics-and-biases 
psychology but with more contested examples of cognitive 
illusions (see Margalit, 1986; Sturm, 2012). 


9. Kant distinguishes several meanings of Vernunft (cf. the rel- 
evant entries in Willaschek, 2015). Only in the broadest sense 
of Vernunft, in which it is the “highest faculty of the mind” and 
opposed to experience (Kant, 1781/1787, A835/B863), is it true 
that understanding is part of reason. 


10. In fact, each special moral duty is a categorical imperative. 
When Kantians speak of the categorical imperative, they refer 
to the principle underlying all moral norms. 


11. Today, many argue that logic not only prescribes how to 
reason well but is even constitutive of rationality (as an interpre- 
tation of, e.g., Kant’s position, see Hanna, 2006). One may also 
claim, more weakly, that only some logical rules are constitutive 
of rationality (Cherniak, 2001); rationality is constituted not 
only by logic but also by other formal systems, especially proba- 
bilistic and decision-theoretic rules (Stein, 1996; for historical 
background, see Erickson et al., 2013); not only logic but also 
the semantics and pragmatics of language matter for good rea- 
soning (Grice, 2001; this is also accepted by the bounded ratio- 
nality program of Gigerenzer, Todd, & ABC Research Group, 
1999); or logic is not constitutive of rationality at all (Mercier & 
Sperber, 2017). 


12. Ido not use Frege’s logical notation but the more familiar 
Hilbert-Ackermann notation. 
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1.2 The Rationality Debate in the Psychology of Reasoning: A Historical Review 


Jonathan St. B. T. Evans 


Summary 


For most of its history, the psychology of reasoning 
assessed people’s ability to evaluate logical arguments 
without formal training, on the assumption that logic 
provided the standard for rational reasoning. Numerous 
experiments reported from the 1960s onward showed that 
logical errors, cognitive biases, and content-dependent 
reasoning were common in university student popula- 
tions. This led to great debate about human rationality, in 
which philosophers were involved. This chapter traces 
the history of that debate, in which arguments were 
made (a) that standard logic is not necessarily the cor- 
rect norm for human reasoning, (b) that apparent errors 
may reflect misinterpretations of the task or instruc- 
tions, and (c) that the experiments were artificial and 
unrepresentative. Ultimately, most psychologists were 
unwilling to label participants as irrational, and toward 
the end of the 20th century, a number of them formed a 
“new paradigm” psychology of reasoning, in which use 
of prior beliefs and expressions of uncertainty in reason- 
ing is regarded as rational. 


1. Reasoning and Rationality 


Traditionally, the psychology of reasoning has consisted 
of giving people logic problems to solve and compar- 
ing their deductions with the dictates of formal logic. 
The first such experiments were published early in the 
20th century (Wilkins, 1928; Woodworth & Sells, 1935) 
and immediately demonstrated some basic findings that 
were to occupy a large field of study in the later part of 
the century (see Evans, Newstead, & Byrne, 1993). First, 
people make many logical errors. In particular, they fre- 
quently endorse fallacies, that is, arguments whose con- 
clusions could be true, given the premises, but are not 
necessary and valid inferences. People are also system- 
atically biased by both the linguistic form of the prem- 
ises and their content or meaning. That is to say, they 


respond to factors unrelated to the logic of the problems. 
Given a philosophical heritage that decreed that logic 
was the basis for rational human thought (Henle, 1962), 
the rationality debate in the psychology of reasoning 
was always likely to happen. It eventually exploded into 
life in the 1980s and has never really subsided. It did, 
however, cause many researchers in the field radically 
to rethink their approach, eventually leading to a “new 
paradigm” in 21st-century psychology of reasoning. 
How and why the field got from its origins to this new 
paradigm is a story I shall tell in this chapter. 

Although the concept of rationality has become much 
more nuanced in recent publications, a basic distinction 
that is commonly made is between instrumental and 
epistemic rationality. Instrumental rationality is served 
when organisms achieve their goals, and it is evident 
that both humans and other animals need to be rational 
in this sense if they are going to survive and reproduce. 
Epistemic rationality is the acquisition and maintenance 
of true beliefs about the world. Epistemic rationality is 
clearly subservient to instrumental rationality, but in 
higher organisms and especially humans, the mainte- 
nance of accurate knowledge about the world around 
us is required for rational actions that will achieve our 
goals. For example, animals must know how and where 
to obtain food and water. The notion of epistemic ratio- 
nality is much more developed for us human beings 
with our unique possession of language, explicit belief 
systems, and capacity for reasoning. Deductive reason- 
ing is a key concept. If the fundamental beliefs we hold 
are true, then so too will be any others that we deduce 
from them in a logically valid manner—hence, the tradi- 
tional emphasis on logic as the basis of rational thought. 

The modern psychology of reasoning dates from the 
1960s and especially the pioneering studies of the British 
psychologist Peter Wason (see Wason & Johnson-Laird, 
1972). The belief that logic provided the basis for ratio- 
nal thinking was dominant at that time and not simply 
because psychologists were influenced by philosophical 


88 


tradition (Henle, 1962). It was also the heyday of Jean 
Piaget, whose theoretical work was enormously influen- 
tial. Piaget had proposed that children’s thinking and 
reasoning develop through a series of stages culminating 
in “formal operations” by adulthood (Inhelder & Piaget, 
1958). In other words, Piaget proposed not only that 
logic should be the basis for adult human thought but 
that this actually was so. By contrast, Wason accepted 
logic as the normative framework but argued strongly 
that people were illogical and therefore irrational. His 
ideas presaged the rationality debate to come. 


2. Origins of the Rationality Debate: The Case 
for Irrational Reasoning 


2.1 Bias and Error in Deductive Reasoning 

As already mentioned, the field of deductive reasoning has 
early origins. Prior to Wason, most studies used syllogis- 
tic reasoning tasks, based on the simple system of logic 
developed by Aristotle. Even the earliest studies showed 
evidence of cognitive bias. For example, Woodworth and 
Sells (1935) showed an “atmosphere effect” in abstract syl- 
logistic reasoning, the first in a number of accounts of why 
people often accept invalid conclusions due to syntactic 
properties of the premises or their arrangement (see Evans, 
Newstead, et al., 1993, chapter 7). Consider the example 


(1) Some A are B 
Some C are B 
Therefore, some A are C 


Due to the atmosphere effect, it was argued, people are 
more likely to accept this conclusion as valid than, say, 
“All A are C,” which has a different quantifier. Both con- 
clusions are, of course, logically invalid. (Consider, for 
example, “Some men are golfers; some women are golf- 
ers; therefore, some men are women.”) 

Mental model theorists later described a “figural bias,” 
which operates through the order of terms (Johnson- 
Laird & Steedman, 1978). Consider the case 


(2) Some A are B 
Some B are C 


Therefore, some A are C 


The terms A, B, and C here run in the same order in prem- 
ises and conclusions, leading to more inferences of this 
type than if the order is as shown in (1). Perhaps the most 
sophisticated model of bias in abstract syllogistic reason- 
ing is that presented by Chater and Oaksford (1999). They 
showed that people tend to choose the form of the least 
informative premise as that of the conclusion—the “min- 
heuristic.” Hence, the validity judgments of participants, 
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normally university students, are often influenced by 
factors other than logical reasoning. 

From the very beginning, it was also discovered that 
when content-rich syllogisms are used, the prior beliefs 
that people hold influence their decisions. In particular, 
as first shown by Wilkins (1928), there is a tendency for 
people to endorse the validity of syllogisms because they 
believe the conclusion to be true, now generally known as 
the belief bias effect. This was followed by various studies of 
questionable methodology, reviewed by Evans, Barston, 
and Pollard (1983). These authors, however, established 
the reality of the belief bias phenomenon with improved 
methodology leading to the modern study of the effect, 
with many studies published (for recent examples, see 
Dube, Rotello, & Heit, 2010; Klauer, Musch, & Naumer, 
2000; Trippas, Handley, & Verde, 2013). Consider the fol- 
lowing two syllogisms used by Evans et al. (1983): 


(3) No addictive things are inexpensive 
Some cigarettes are inexpensive 


Therefore, some addictive things are not cigarettes 


(4) No millionaires are hard workers 
Some rich people are hard workers 


Therefore, some millionaires are not rich people 


The two syllogisms have the same form, and neither is 
valid. However, Evans et al. found that 71% of partici- 
pants declared (3) a valid syllogism, compared with only 
10% for (4). The difference, of course, is that the conclu- 
sion to (3) is believable and that to (4) unbelievable. The 
importance of this basic finding for the present purposes 
is quite simple: despite being told clearly to assume the 
premises and decide whether the conclusion follows 
logically, participants are quite unable to ignore their 
(logically irrelevant) actual beliefs. By the time I wrote 
my first book reviewing the psychology of deductive rea- 
soning (Evans, 1982), much evidence had been accumu- 
lated in the psychological experiments for illogicality, 
systematic biases, and a strong influence of prior belief 
on reasoning, and many similar findings have contin- 
ued to be reported since. 

One influential approach has been the proposal that 
people reason by use of mental logics comprising infer- 
ence rules, which are instantiated for particular contexts 
(Braine & O’Brien, 1991, 1998; Rips, 1983, 1994). However, 
this approach attracted strong criticisms (Johnson-Laird, 
1983; Oaksford & Chater, 1991), and Philip Johnson- 
Laird and colleagues went on to propose a highly influen- 
tial alternative view that people reason by imagining and 
eliminating possibilities. This led to a major research pro- 
gram (see Johnson-Laird, 2006; Johnson-Laird & Byrne, 
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1991) with a very large number of published studies. 
However, mental model theorists have mostly stuck quite 
closely to propositional logic as a normative account (e.g., 
Byrne & Johnson-Laird, 2009; Johnson-Laird & Byrne, 
2002), in contrast with researchers in the new paradigm, 
discussed later. Mental model theorists have, however, 
investigated cognitive biases and demonstrated cognitive 
illusions (Johnson-Laird & Savary, 1999). Their approach 
to rationality is that people are rational in principle but 
fallible in practice, mostly due to limitations in working 
memory capacity. This complies with the idea of bounded 
rationality proposed by Herbert Simon (1982). Johnson- 
Laird (2006) has further clarified two fundamental bases 
for human rationality within the mental models frame- 
work: first, our ability to search for and identify counter- 
examples, which demonstrate invalid arguments, and, 
second, our ability to understand and reason about what 
is true or false. He maintains that rationality does not 
depend upon any form of internalized logical rules, pro- 
viding a radical alternative account of epistemic rational- 
ity and deductive competence. 


2.2 Peter Wason and the 2 4 6 and Selection Tasks 
Peter Wason was a British psychologist who published 
most of his work in a 25-year period starting in the late 
1950s. He is particularly famous for the invention of two 
fiendish reasoning tasks that appeared simple but led 
most of his participants into logical errors. Of equal sig- 
nificance for this historical review are the comments that 
he wrote about these tasks, now rarely read and noted by 
modern students of the field. His first invention was the 
“2.4 6” task (Wason, 1960), which continues to be studied 
to the present day (for a full history, see Evans, 2016b). 
For a highly cited paper, it is actually quite short, only a 
few pages long; it contains a study that can scarcely be 
described as an experiment as it has only one condition 
with no controls or comparisons. As with the first presen- 
tation of his even more famous four-card selection task 
(Wason, 1966), the reader is simply invited to compare the 
poor performance of his participants to their prior expec- 
tation that the task ought to be simple and straightforward. 
While the selection task enjoyed a “heyday” between 
1980 and 2000, in which it was the focus of a number of 
major theoretical papers, citations of Wason’s work on this 
continue to increase to the current day (Evans, 2016a)— 
remarkable when one considers that Wason’s last major 
article was published more than 30 years ago. 

Wason considered his tasks to be “deceptively simple,” 
by which he presumably meant that they look easy while 
being anything but. The 2 4 6 task is so called because 
participants are told that the experimenter has a rule in 
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mind that classifies triples of whole numbers, an example 
of which is 2 4 6. They were to discover the rule by offering 
triples of their own and being told whether or not they con- 
formed to the experimenter’s rule. The actual rule was any 
ascending sequence, much more general than the delib- 
erately biasing example of 2 4 6 suggests. Two unusual 
features of the task were that participants were required to 
write down the reasons for their choice of triples and also, 
if they announced the wrong rule, to continue with the 
task until they gave up or a time limit was reached. What 
happened was that people formed more specific hypoth- 
eses, such as “ascending with equal intervals,” and got 
repeated positive feedback from testing triples that con- 
formed to it (e.g., 1 3 5, 10 30 50, etc.). Few people solved 
this problem without announcing wrong rules and some 
not at all. Wason also published example protocols that 
showed how people became convinced their hypotheses 
were correct and often reformulated them in other words, 
so that they announced the same wrong rule more than 
once. 

The 2 4 6 task does not fit the standard psychological 
paradigm for studying deductive reasoning and perhaps 
has elements of induction (rule discovery) or abduction 
(reasoning to best explanation), although it also requires 
hypothetico-deductive reasoning. The selection task also 
falls outside of the standard paradigm, although it is 
generally studied by deduction researchers. In the origi- 
nal task (Wason, 1966), participants were told that a set 
of cards each had a letter on one side and a number on 
the other. They were also told that a rule applied to these 
four cards and might be true or false: 


If a card has a vowel on one side then it has an even 
number on the other side. 


They were then shown four cards lying on a table, whose 
facing sides displayed a vowel, a consonant, an even 
number, and an odd number, for example, 


EP47 


Their task was to choose those cards, and only those 
cards, that needed to be turned over in order to decide 
whether the rule was true or false. Wason maintained 
(and most subsequent researchers of the task agreed) 
that the correct choices were the vowel and the odd 
number, E and 7 in the example given. This is because, 
logically, the conditional statement can only be false if 
a card is found with a vowel on one side that does not 
have an even number on the other. Very few people gave 
this response—later estimates of solution rates are usu- 
ally around 10%. Most people chose only the vowel or 
the vowel and the even number (e.g., E and 4). 
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Wason believed that both tasks showed that partici- 
pants had a verification bias, contrary to the then fash- 
ionable view of Karl Popper (1959) that the rational 
approach to hypothesis testing was falsification. Verifi- 
cation or confirmation bias has not withstood the scru- 
tiny of later researchers as an account of either task, but 
the details are not relevant for current purposes. Of more 
concern is Wason’s assertion that failure on these tasks 
demonstrated that people were fundamentally irratio- 
nal. The case was made in a number of places, including 
an influential book of the time (Wason & Johnson- 
Laird, 1972). People were illogical and therefore did not 
develop formal operational thought as Inhelder and 
Piaget (1958) had proposed; they verified, and failed to 
falsify, their hypotheses and hence violated Popper’s pre- 
scriptions for scientific thinking. Wason drew the stron- 
gest possible conclusion from his studies of the selection 
task: 


The selection task reflects [a tendency towards irrationality 
in argument] to the extent that subjects get it wrong. .. . It 
could be argued that irrationality rather than rationality is 
the norm. People succumb all too readily to logical failures. 
(Wason, 1983, p. 59) 


Wason was being controversial and knew it. He was 
well aware, for example, of a philosophical traditional 
that made logical reasoning the centerpiece of rational 
thinking, as he included the famous paper of Henle 
(1962) in one of his edited collections. Henle asserted 
that apparent failures of logical reasoning were due to 
misinterpretations, additions of unstated premises, or 
“failures to accept the logical task.” Wason railed against 
this tradition as well as the overbearing influence of 
Piaget in the psychology of that period (Wason, 1977). 
He believed—with clear justification at the time—that 
he had discovered that people were fundamentally irra- 
tional in their reasoning. 

The selection task went on to have a long and distin- 
guished history of study (Evans, 2016a), especially since 
it was discovered quite early on that versions using real- 
istic content could be much easier to solve. This led ini- 
tially to a belief in a general thematic facilitation effect, 
but the story proved a good deal more complicated as 
later research unfolded (see Evans, Newstead, et al., 1993; 
Evans & Over, 2004). Not all realistic versions facilitate, 
and most versions that do switch from indicative to 
deontic logic (Manktelow & Over, 1991). That is, the rules 
tested are typically permissions and obligations such as 
“Tf you drink alcohol in a bar you must be over 18 years 
of age,” and the task set is to check whether or not the 
rule is obeyed rather than whether it is true or false. I will 
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return to this when discussing evolutionary approaches 
later in the chapter. 


2.3 The Influence of Kahneman and Tversky 

In the early 1970s, when belief in irrationality was at its 
peak in the psychology of reasoning, the consensus view 
in a different tradition—the study of judgment and deci- 
sion making (JDM)—was still that people were essentially 
rational. This changed dramatically when Amos Tversky 
and Daniel Kahneman started to publish their famous 
papers, which launched the heuristics and biases tradi- 
tion (e.g., Tversky & Kahneman, 1974). Many reasoning 
researchers also paid attention to the JDM literature, so 
this work also had a clear influence on the rationality 
debate in the psychology of reasoning. The combined 
impact of studies in both fields helped to stimulate the 
historically important critique of Cohen (1981). 

Prior to Kahneman and Tversky’s work, the field of 
judgment and decision making had enjoyed a belief 
in rationality similar to that promoted by Henle and 
Piaget. While it was conceded that people had calibra- 
tion issues, they were generally considered to be good 
intuitive statisticians (Peterson & Beach, 1967). Eco- 
nomic decision theory had exerted a big influence on 
the study of decision making thanks to Ward Edwards 
(e.g., 1961) with attempts to propose subjective ver- 
sions of both probability and utility to fit the model 
of the “rational man” of decision theory. Tversky and 
Kahneman blew this cozy consensus away with a series 
of papers demonstrating systematic biases in statistical 
judgment and, later, decision making (Kahneman & 
Tversky, 1979). Kahneman would later receive the Nobel 
Prize for Economics for this work, Tversky being by then 
deceased and hence ineligible. I have no space to review 
this work in detail here but note that the fundamental 
idea was that people relied on simple heuristics such as 
representativeness (Kahneman & Tversky, 1972) and 
availability (Iversky & Kahneman, 1973) when making 
probability judgments. While sometimes helpful, these 
heuristics frequently led to systematic biases, as they 
demonstrated in a series of ingenious experiments. This 
set in train a tradition known as “heuristics and biases,” 
which was studied mostly by other researchers as time 
went on, with collections of papers later published in 
influential books (Gilovich, Griffin, & Kahneman, 2002; 
Kahneman, Slovic, & Tversky, 1982). 

Unlike Wason, Kahneman and Tversky generally 
avoided direct claims that they were showing people 
to be irrational, but their many admirers and followers 
were less restrained. There was great interest in this work 
in business schools, which increased when they added 
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prospect theory (Kahneman & Tversky, 1979; Tversky & 
Kahneman, 1992) as an alternative to the traditional 
teaching of economic decision theory. By the end of the 
1970s, the idea that people were fundamentally irratio- 
nal in their reasoning, judgment, and decision making 
was taking a strong hold and was no longer confined to 
the journals of academic psychology. However, strong 
reaction from the defenders of human rationality was 
soon to arrive. 


3. The Critique of Cohen (1981): Three Arguments 
for Rationality 


Jonathan Cohen (1981) was by no means the only author 
to react against the emerging view of irrational think- 
ing in the psychology of reasoning and judgment, but 
his paper was the most influential both at the time and 
for many years to follow. Cohen was a philosopher and 
hence employed a priori rather than empirical argu- 
ments. He posed the question, “Can human irrationality 
be experimentally demonstrated?” in his title and provided 
the answer in his text: an emphatic “no.” This position 
and ones like it have later been dubbed “Panglossian” 
(see, e.g., Stanovich, 2011). However, Cohen made sev- 
eral strong arguments, which I later described (Evans, 
1993) as falling into three broad categories: 


1. The normative system problem 
2. The interpretation problem 
3. The external validity problem 


Since these three basic arguments have also arisen in 
much more recent debate about rationality, I will take a 
little time to explain them. The normative system problem 
is that when participants are judged to be right or wrong, 
this is done by reference to a standard normative theory, 
which itself may be subject to argument and alternatives. 
For example, Wason adopted Popper’s falsificationist 
philosophy of science as his normative system when he 
described verification bias on the 2 4 6 and selection tasks. 
Popper argued that lawful scientific statements were strict 
universals and could hence never be logically verified, 
but they could be falsified. Popper’s work was recent and 
“hot” in the early 1960s when Wason was writing but has 
been severely challenged since, with some contemporary 
philosophers and psychologists preferring a Bayesian phi- 
losophy of science, in which belief in scientific theories is 
gradually acquired or lost as evidence accumulates (How- 
son & Urbach, 2006). 

Although most psychologists still accept Wason’s state- 
ment of the correct solution to the selection task, it has 
been famously challenged (Oaksford & Chater, 1994). The 
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“alternative norms” problem later became a major issue 
for debate in the field (e.g., Stanovich, 1999): you do not 
need to be a Panglossian to recognize the ambiguity of 
norms. Much more recently, Elqayam and Evans (2011) 
discussed the normative system problem at length in 
order to argue their case that normative rationality has no 
useful part to play in cognitive psychology. They argued 
that ambiguous and alternative norms frequently arise in 
the literatures on reasoning and decision making. Indeed, 
it was the questioning of standard binary logic and the 
material conditional that led to the new paradigm psy- 
chology of reasoning discussed at the end of this chapter. 
So the problem here is that if people are to be judged to be 
reasoning incorrectly and therefore to be irrational, do we 
actually have a clear and agreed standard for what is right 
and wrong in the first place? 

The second type of objection made by Cohen and 
others (e.g., Henle, 1962; Smedslund, 1970, 1990) is that 
participants may not interpret the problem in the way that 
the experimenter intends and assumes. If, for example, 
they add or omit premises in deductive reasoning, then 
the conclusion that follows from their interpretation 
cannot be compared to the one prescribed by the experi- 
menter. In the statistical reasoning literature, a good 
example is provided by the conjunction fallacy originally 
reported by Tversky and Kahneman (1983) and much 
investigated thereafter: people make an apparent logical 
error by judging that the probability of A&B is greater 
than the probability of A. In the famous Linda problem, 
Linda is described in a way that provokes the stereotype 
of a feminist. Participants then appear to judge as more 
likely that she is a bank teller and a feminist than just a 
bank teller, which is logically impossible. However, the 
context might suggest that the “bank teller” option is 
really a bank teller who is not a feminist. With this inter- 
pretation, there is no error. A number of authors have 
proposed interpretational explanations of the conjunc- 
tion fallacy (see Moro, 2011). 

Cohen also suggested that psychological experiments 
can be artificial and unrepresentative, dismissing the 
Wason selection task as a “cognitive illusion.” I call this 
the external validity problem. Actually, the 2 4 6 task is a 
better example, as Wason’s claim that it showed a veri- 
fication bias was challenged almost immediately (Weth- 
erick, 1962). Wetherick’s objections were essentially 
correct, if largely ignored at the time. The paper was not 
even cited by Wason and Johnson-Laird (1972), whose 
well-read book popularized Wason’s original view of the 
task. However, from the 1980s onward, other authors 
made similar arguments, which are now widely accepted 
(see Evans, 2016b). The general argument is that while 


92 


Wason showed a preference for positive hypothesis test- 
ing, this only has the effect of repeated confirmation 
due to the fact that participants have been induced to 
adopt a hypothesis much more specific than the actual 
rule. Moreover, it has been argued that positive test- 
ing of hypotheses is generally an adaptive and effective 
strategy in science (Klayman & Ha, 1987) and that the 
adoption of a verification or falsification attitude in sci- 
entific hypothesis testing makes no practical difference 
to the effectiveness of a test (Poletiek, 2001). 

In the case of the 2 4 6 task, it does seem that the origi- 
nal experiment suggested a misleading interpretation and 
that it was indeed unrepresentative of real life. However, 
of Cohen’s three arguments, the external validity prob- 
lem is the weakest. It is implausible to suggest that psy- 
chologists consistently contrive to demonstrate errors 
and biases that only occur as laboratory tricks. As a bias 
researcher myself, I can attest that there is no conspiracy 
among psychologists to denigrate human rationality. In 
the heuristics and biases tradition, in particular, most 
of the cognitive biases discovered originally on artificial 
laboratory tasks have been shown to apply in real-world 
contexts and with expert groups (Gilovich et al., 2002; 
Kahneman et al., 1982). That is not to say, of course, that 
there may not be a number of laboratory demonstrations 
of biases that are, nevertheless, misleading. 


4. The Great Rationality Debate 


Following Cohen’s critique, psychologists started to 
stake out various positions on the topic of rationality in 
reasoning and decision making, most of which were well 
established by the end of the 20th century. This has been 
labeled the “Great Rationality Debate” by Keith Stano- 
vich (2011), who in an earlier book (Stanovich, 1999) 
had already categorized them as falling into three main 
camps, which he termed “Panglossian,” “Meliorist,” and 
“Apologist.” Panglossians, like Cohen, essentially argue 
that irrationality is not possible and there can be no gap 
between descriptive and normative accounts of human 
behavior (see Stanovich, 1999, p. 5). Meliorists claim 
that, while often convergent, such accounts can also 
diverge. People can fail to attain normative rationality 
due to lack of education, cognitive capacity, or rational 
attitude, but much of this divergence can be addressed 
by education and training. Meliorists include Jonathan 
Baron (1985) and Stanovich himself. Finally, Apolo- 
gists attempt to explain why performance often fails to 
match normative standards and give much weight to 
computational limitations, in the “bounded rationality” 
tradition of Simon (1982). 
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Not all authors seem to fall into Stanovich’s catego- 
ries, including myself, as I have consistently questioned 
the value of normative theory itself in the study of rea- 
soning since my earliest papers on the topic (Evans, 
1972). Authors such as myself may be termed “descrip- 
tivists” (Elqayam & Evans, 2011). Ina collaboration with 
the philosopher David Over, I suggested that normative 
rationality (conforming to a normative theory) may 
often diverge from instrumental rationality (acting in 
such a way as to achieve one’s goals)—see Evans and Over 
(1996). (This view was strongly contested by Stanovich, 
e.g., 1999, who regards normative theory as a descrip- 
tion of instrumental rationality.) Evans and Over (1996, 
chapter 2) strongly critiqued decision theory as lacking 
the boundaries and constraints necessary for compari- 
son with actual human choices. In a later book (Evans 
& Over, 2004), we challenged the binary logic that has 
traditionally been used to provide norms for deductive 
reasoning. We also picked out the case of belief bias as 
being a possible artifact of experimentation (Cohen’s 
external validity problem), since in the real world, it is 
rational to reason from all relevant belief (Evans, Over, 
& Manktelow, 1993). However, when I later teamed up 
with Shira Elqayam to present a critique of normative 
rationality in Behavioral and Brain Sciences (Elqayam & 
Evans, 2011), it became apparent from the commentar- 
ies it elicited that many authors disagree with us. The 
majority view appears to be that normative theory does 
have an important role to play in the descriptive study 
of reasoning and decision making, and it continues to 
play a central role in the Great Rationality Debate. 


4.1 Evolutionary and Ecological Approaches 

While the philosopher Cohen started the rationalist 
backlash against research on cognitive biases, the cause 
of humanity was taken up later by psychological writers. 
Many assumed that human beings would be adapted to 
their environment, which may occur by evolution, learn- 
ing, or a combination of both. Gerd Gigerenzer emerged 
in the early 1990s and launched a series of attacks on the 
work of Tversky and Kahneman as providing an inac- 
curate picture of irrationality. This eventually led to an 
exchange in the pages of Psychological Review (Gigeren- 
zet, 1996a; Kahneman & Tversky, 1996). One of Gigeren- 
zer’s arguments was that people are able to reason with 
frequencies rather than probabilities and that appar- 
ent cognitive biases could be made to disappear when 
frequencies were used in the word problems instead 
(Gigerenzer, 1991; Gigerenzer & Hoffrage, 1995). This 
was supported by the work of the evolutionary psychol- 
ogists Cosmides and Tooby (1996; see also chapter 10.6 
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by Cosmides & Tooby, this handbook) but resulted in a 
flurry of papers from skeptical authors disputing their 
explanation, based on Darwinian algorithms, of why fre- 
quency formats make the problems easier. Critics sug- 
gested that these formats cued simple mental models 
that facilitated reasoning for reasons that have nothing 
to do with evolution (see Barbey & Sloman, 2007, for an 
extended review and discussion of this debate). 

Leda Cosmides and John Tooby set out a more gen- 
eral argument for massive modularity (Cosmides & Tooby, 
1992; Tooby & Cosmides, 1992), according to which the 
mind comprises specialized self-contained modules. 
Cognitive modularity was originally proposed by Fodor 
(1983), who argued that many such modules are con- 
tained in the mind, for example, to process language 
or to explain much of the working of the visual system 
(although many authors, including Cosmides and Tooby, 
use the term without subscribing to all of his detailed 
definitions). Importantly, Fodor argued that there must 
also be a general-purpose reasoning system, something 
that the massive modularity hypothesis denied. Cos- 
mides and Tooby argued that evolution could only pro- 
duce specialized and not general reasoning systems, a 
highly controversial position. Fodor (2000, 2001) was 
among their many critics, and a more general cover- 
age of the debate can be found in Over (2003). In the 
psychology of reasoning, these arguments were first 
applied to the Wason selection task. As mentioned 
earlier, the task is much easier when people are asked 
to check whether permissions or obligations have been 
complied with. Cosmides (1989) argued that these ver- 
sions of the task comprised social contracts, for which we 
have evolved a cheater-detection module. In later work, 
this was extended to include a hazard-avoidance mod- 
ule (Fiddick, Cosmides, & Tooby, 2000), which applied 
to other contexts in which the task becomes easy. Their 
work on this topic was also supported by Gigerenzer and 
Hug (1992). 

Again, all these claims proved highly controversial 
and were subject to critiques of methodology and the 
postulation of rival theoretical accounts that did not 
depend upon evolutionary mechanisms. The relevance 
to the rationality debate is the argument that people 
have an inherent rationality in their reasoning provided 
by their genes but that it only operates in contexts of evo- 
lutionary relevance. One problem with this approach, 
pointed out by Stanovich (2004), is that evolutionary 
and individual rationality can diverge. For example, the 
craving for fat and sugar that made great evolutionary 
sense to our ancestors has become a cause of obesity and 
health problems in the modern world. Modern humans 
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can also frustrate the goals of the genes, for example, 
by enjoying sex with contraception. Stanovich (2004) 
claimed that human beings are the one species that the 
genes lost control of, as we can think for ourselves and 
pursue our own goals. 

Gigerenzer and colleagues developed a major research 
program based on “fast and frugal heuristics,” which are 
taken from an evolutionary toolbox similar to the mod- 
ular mind of Cosmides and Tooby (Gigerenzer, 1996b; 
Gigerenzer & Todd, 1999). In contrast with the heuristics 
of Kahneman and Tversky, these are portrayed as mak- 
ing us smart without the need for much thought. In fact, 
Gigerenzer claims that ignorance often trumps knowl- 
edge and that we are better off relying on gut feelings 
than engaging in reasoning (Gigerenzer, 2007), ironi- 
cally employing masterful reasoning to make his case! 
Again, we see the denial of any useful role for a general- 
purpose reasoning system, which puts this approach in 
direct conflict with the “dual-process theories” of rea- 
soning discussed later. In fact, Gigerenzer is a strong 
critic of dual-process theory (Gigerenzer, 2011; Kruglan- 
ski & Gigerenzer, 2011). Evolutionary arguments have 
also been made to explain our reasoning facility without 
attributing to it the importance that is usually attached 
to it. For example, it has been proposed that reasoning 
evolved for purposes of argumentation rather than prob- 
lem solving (Mercier & Sperber, 2011), although a more 
complex story has emerged in later development of this 
account (Mercier & Sperber, 2017). 

Evolutionary rationality is closely related to the idea 
of ecological rationality, which comes from a slightly dif- 
ferent tradition. Rational analysis is a tradition in cogni- 
tive psychology started by John Anderson (1990, 1991). 
It has been applied to the psychology of reasoning and 
decision making in a major program led by Mike Oaks- 
ford and Nick Chater (Oaksford & Chater, 1998, 2007), 
one of the roots of the new paradigm psychology of rea- 
soning discussed at the end of this chapter. It is grounded 
in ecological rationality and related to other psycho- 
logical work in this tradition (Gibson, 1979; Hammond, 
1966). The approach takes it as axiomatic that behav- 
ior is adapted to the environment and that therefore an 
analysis of the environment itself is required in order 
to see what is the optimal way of operating within it 
(a computational-level account, in the terminology 
of Marr, 1982). Normative theory, while important, is 
subservient to this ecological rationality. As Oaksford 
and Chater (2007) put it, “According to this viewpoint, 
formal rational principles relate to explaining every- 
day rationality because they specify the optimal ways 
in which the goals of the system can be attained in a 
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particular environment” (p. 32). They also make it per- 
fectly clear that if a standard normative theory, such as 
binary or propositional logic, fails to achieve that task, 
then it must be discarded and replaced. Much of their 
research, starting with their alternative account of the 
selection task (Oaksford & Chater, 1994), has consisted 
of providing alternative normative accounts that render 
the actual observed behavior of their participants nor- 
matively correct, in the spirit of Cohen’s first argument 
for rationality. Their position evolved to one of arguing 
that human rationality is based upon Bayesian decision 
theory rather than logic. 


4.2 Dual Processes, General Intelligence, 

and Individual Differences 

Dual-process theories have become highly popular in 
cognitive and social psychology but have diverse ori- 
gins and many different implementations (Evans, 2008; 
Evans & Frankish, 2009). They are also highly contro- 
versial and have been the subject of many criticisms (see 
Evans & Stanovich, 2013). What all have in common 
is the assertion that there are two fundamentally dif- 
ferent kinds of human thought, one fast and intuitive 
and the other slow and reflective. These are sometimes 
referred to as “System 1” and “System 2” (Kahneman, 
2011; Stanovich, 1999), although both Stanovich and I 
now prefer the more neutral reference to “Type 1” and 
“Type 2” processing in our current writing. Modular cog- 
nitive processes would be automated and therefore Type 
1, but such processing can also be based upon associa- 
tive or implicit learning (Reber, 1993) or on the automa- 
tion of knowledge that was once explicit and processed 
in a Type 2 manner. Type 2 processing is now generally 
thought to involve a central general-purpose working 
memory system of the kind described by Baddeley and 
his coworkers (Baddeley, 2007) and is related to the con- 
cept of general intelligence. Hence, dual-process theory 
strongly asserts the existence of a general-purpose rea- 
soning system, in stark contrast with the evolutionary 
approaches discussed above. 

Traditionally, dual-process theory has largely explained 
cognitive biases in terms of Type 1 processing and cor- 
rect reasoning or decision making in terms of Type 2 
processing (Evans, 1989; Kahneman, 2011; Stanovich, 
1999). However, this is oversimplified as intuitions can 
be sound and reasoning flawed. Type 2 processing, for 
example, can go awry because of insufficient cognitive 
capacity, lack of relevant education or “mindware,” or a 
failure to adopt a rational attitude in which intuitions are 
carefully checked (Evans, 2007; Evans & Stanovich, 2013; 
Stanovich, 2011). Nevertheless, much has been made of 
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the fact that general intelligence correlates not only with 
working memory capacity but also with normatively cor- 
rect answers on a wide range of laboratory reasoning and 
decision tasks (Stanovich, 1999, 2011; Stanovich, West, & 
Toplak, 2016). However, the same research program has 
always shown that another individual difference factor 
has a significant, if weaker, influence on correct perfor- 
mance. This is rational thinking disposition, assessed by a 
variety of psychometric scales. In essence, it appears that 
some people are inclined to make quick decisions based 
on intuitions and others to check them out by more care- 
ful reasoning. Since most laboratory tasks are designed to 
minimize the value of Type 1 processing (previous expe- 
rience is unhelpful) and to maximize the role of Type 
2 processing (problems are novel and load on working 
memory), a rational thinking disposition aids solution. 
Stanovich has discussed rationality at length in his 
several books on this topic. In the first of these (Stano- 
vich, 1999), he rejected the argument that rationality is 
revealed by alternative normative accounts, as in the 
work of Oaksford and Chater. He points out that on most 
of the tasks studied, general intelligence correlates with 
the standard normative solution. For example, even on 
the very difficult abstract form of the Wason selection 
task, the few people who do manage to solve it have 
higher IQs than those who do not (Stanovich & West, 
1998). In this and later books, he develops the idea that 
rationality should not, however, be equated with IQ, due 
to the strong influence of rational thinking dispositions: 
there should therefore be a Rationality Quotient, or 
“RQ,” that takes both into account (Stanovich, 2009). In 
a recent major project, Stanovich and his collaborators 
explored the psychometric properties of a large range 
of laboratory tasks in order to provide the foundation 
for such an RQ test (Stanovich et al., 2016), called the 
“CART.” The CART still loads heavily on general intelli- 
gence, however, and currently lacks both the item analy- 
sis and validation studies that would be required to turn 
it into a marketable psychometric instrument. 
Dual-process theory is currently undergoing some- 
thing of a crisis due to a series of results that show that 
correct answers on many of the tasks may be given 
quickly and intuitively and do not benefit from slower 
reflection (see a number of papers in the collection edited 
by De Neys, 2017). Recent findings also suggest that 
people of higher IQ have better intuitions, or Type 1 pro- 
cesses, than those of lower intelligence. This is leading 
some authors, previously sympathetic to the theory, to 
question the traditional dual-process account in which 
better normative performance on tasks and by individu- 
als is achieved by Type 2 processing. I personally think 
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that some of this pessimism is premature as much of 
the research is based on relatively simple tasks in which 
“logical intuitions” are less surprising than they would 
be on tasks that load heavily on working memory. 


5. The Rejection of Binary Logic and the New 
Paradigm Psychology of Reasoning 


The rationality debate in the psychology of reasoning 
has produced a tangible result: a new paradigm. The tra- 
ditional method, dubbed the “deduction paradigm” by 
Evans (2002), provides no instruction in logic and rejects 
participants who have such training. The idea is to see if 
people are inherently logical in their reasoning. Gener- 
ally, they are given the premises of an argument, which 
they are told to assume are true, and asked if a partic- 
ular conclusion necessarily follows. Their decisions are 
then compared with those of standard logics such as the 
propositional calculus. By the end of the 20th century, 
this paradigm was creaking under the weight of a mass 
of findings showing that people (a) made many logical 
errors, (b) had systematic cognitive biases, and (c) fre- 
quently reasoned on the basis of actual beliefs rather 
than assumed premises. A Kuhnian crisis was looming 
(Evans, 2002). There were also powerful voices, such as 
those of Oaksford and Chater, arguing that the norma- 
tive model of standard logic was simply inapplicable to 
everyday reasoning. 

With the pressure of the rationality debate upon 
them, reasoning researchers were basically faced with two 
choices: (1) defend the deduction paradigm and conclude 
that people were massively irrational or (2) abandon the 
paradigm, especially the use of standard logic as a norma- 
tive system. Many chose the latter and moved toward a 
new paradigm, although the mental model camp put up 
a lot of resistance, as mentioned earlier. A key develop- 
ment came in the study of conditionals when psycholo- 
gists became increasingly aware that many philosophers 
reject the material conditional of propositional logic as 
a description of the ordinary conditional of natural lan- 
guage (Edgington, 1995). In this system, “If p then q” 
means the same thing as “Either not-p or q,” but this 
leads to absurd and paradoxical inferences. Such a condi- 
tional is always true when either p is false or q is true (or 
both). So, for example, “If Hillary Clinton is president of 
the USA, then the UK is at war with France” must be true. 

Also taking the lead from philosophers, psychologists 
became interested in the Ramsey test, which is the idea 
that we evaluate a conditional by imagining p to be true 
and then evaluating g in a thought experiment. This 
was quickly linked with the idea that the probability of 
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the conditional statement should be equal to the condi- 
tional probability, that is, P(if p then q) = P(q| p), known 
as “the Equation.” Psychological evidence was gathered 
in the early 2000s that many people complied with the 
Equation and not with the material conditional when 
judging the likelihood of conditional statements (Evans, 
Handley, & Over, 2003; Oberauer & Wilhelm, 2003; Over, 
Hadjichristidis, Evans, Handley, & Sloman, 2007). David 
Over and I presented a suppositional theory of condi- 
tionals (Evans & Over, 2004) developing the Ramsey test 
into a full-blown psychological account of conditional 
thought. This work, together with the case that Oaksford 
and Chater had been pressing for a decade, led to what 
is now clearly recognized as a new paradigm in the psy- 
chology of reasoning (Elgayam & Over, 2012; Oaksford 
& Chater, 2010; Over, 2009). 

Researchers within the new paradigm agree on the 
rejection both of the traditional methods and of the 
normativity of binary logic. Participants are more likely 
to be asked to reason with uncertain premises or to draw 
uncertain conclusions. For some researchers, pragmatics 
is key, as ordinary reasoning is now recognized as belief 
based. For others, the new paradigm involves an attempt 
to replace binary logic with alternative normative sys- 
tems, such as Bayesian decision theory, probability logic, 
or multivalued logic. Researchers may, however, sub- 
scribe to Bayesianism in a strict or a soft sense (Elqayam 
& Evans, 2013), the latter falling short of a normative 
theory. The new paradigm is a looser concept than the 
old paradigm as there is as yet no real general agreement 
as to its priorities, methods, and theoretical framing 
(Evans, 2012). It is quite clear, though, that the ratio- 
nality debate played a critical role in moving most of 
the field away from the traditional study of logicality in 
human reasoning. 


6. Final Thoughts 


Throughout cognitive psychology generally, there is 
no rationality debate. People are not accused of irra- 
tionality for having poor memories or succumbing to 
visual illusions, for example. Even within the psychol- 
ogy of thinking, cognitive failure does not necessarily 
lead to imputations of irrationality; for example, we do 
not expect that people should be able to compute the 
square roots of large numbers in their heads. The issue 
is pretty much restricted to the study of reasoning and 
decision making—so what makes these topics different? 
The only answer that I can see is the strong influence of 
philosophy. The issue of rationality and the efficacy of 
different normative systems are primarily philosophical 
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issues. However, psychologists have become highly 
engaged with them, to the extent that they have tried to 
approach rationality as an empirical question. The only 
way that this can be done is to observe human reason- 
ing and decision making and compare it with a norma- 
tive standard. But as we have seen, such a path is rife 
with pitfalls. All normative theories are themselves con- 
testable, for example. And should the outcome be that 
people appear irrational, strong arguments can be made 
on evolutionary and ecological grounds that this con- 
clusion must be false. But if the conclusion is prejudged, 
why bother asking the question at all? 

Personally, I think these fields of study should be 
approached like the rest of cognitive psychology: we 
should describe the processes of reasoning and decision 
making and simply try to understand the cognitive and 
neural mechanisms that underlie them. However, this 
is not the popular view and certainly does not reflect 
the history of the psychology of reasoning in which the 
rationality debate has played such a prominent role. If 
the Panglossians are right, then they must agree with 
me on how the topic should be studied, as the issue of 
human rationality can never be decided empirically. In 
reality, however, most psychologists have focused on 
right and wrong answers throughout the history of the 
psychology of reasoning and continue to do so today. 
When people were shown to be intransigent in their 
lack of respect for logic, standard logic was abandoned 
in favor of a new paradigm. People are now seen to con- 
cern themselves with degrees of belief rather than truth 
and to reason with uncertainty. But normativity has not 
gone away in this new paradigm: psychologists continue 
to hunt for new formal descriptions of rational reason- 
ing that might account for the way in which people 
actually think. 

One final thought is on the nature of evolution. We 
are what we are and do what we do because we evolved 
to do so. We became the dominant species on Earth— 
at least by some measures—as a result and certainly the 
only one that can with great power and flexibility design 
its environment to suit itself, rather than evolving to fit 
the environment. Our scientific and cultural achieve- 
ments are extraordinary. But the 21st-century world is 
also striven with war, disease, and starvation, and there 
is every likelihood that the failure to control climate 
change or to manage the horrendous weapons of mass 
destruction we have designed will bring the whole proj- 
ect to a premature end. To me this reflects the nature of 
evolution: it does not optimize, and it does not accord 
with some philosophical ideal of rationality. It is driven 
mostly by what worked in the past and provides no 
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guarantees for the future. The ultimate test of the ratio- 
nality of human beings lies not in the psychologist’s 
laboratory but in the highly uncertain and dangerous 
real world that we inhabit. 
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1.3. Evolution of Rationality 


Gerhard Schurz 


Summary 


Rationality is a normative concept that has two dimen- 
sions: theoretical and practical rationality. We suggest 
evaluating theoretical rationality in terms of its cogni- 
tive success and practical rationality in terms of its pro- 
moting well-being and social cooperation. Based on this 
understanding, the evolution of rationality is investi- 
gated from both a philosophical and a psychological 
point of view. Human cognition has evolved in three 
dimensions: biological, cultural, and individual. Many 
researchers believe that human cognition originates 
from a sociolinguistic coevolution. A second characteris- 
tic is the coevolution of human technology and humans’ 
ability to reason causally. Based on evolutionary consid- 
erations, it is argued that human cognition is based on a 
division of labor between specialized cognitive modules 
and general-purpose nonspecific mechanisms of learn- 
ing and reasoning. 


1. Factual and Normative Aspects of Rationality 


Is it rational to increase human power by means of sci- 
ence and technology? Is it rational to believe in God 
because this increases one’s happiness? The answer to 
these questions cannot solely be based on empirical facts 
and logical reasoning, because the notion of rational- 
ity used in these questions has a normative component. 
Since David Hume (1739/2004), we know that there is 
no logically valid inference from facts to norms.’ The 
notion of rationality that one assumes relies at least par- 
tially on a value-loaded decision. 

Thus, before we start this chapter, we have to explicate 
our assumed concept of rationality. In philosophy, one 
usually distinguishes between theoretical rationality (i.e., 
the rationality of our picture of the world) and practical 
rationality (i.e., the rationality of our norms of action). 
The aim of theoretical rationality is to acquire predic- 
tively and causally or explanatorily relevant information 
about the world. More generally formulated, the goal 


of theoretical rationality is cognitive success. Schurz and 
Hertwig (2019) define the cognitive success of a method 
as the product of its validity (percentage of correct infer- 
ences among all inferences rendered) times its applica- 
bility (percentage of all problems to which the method 
is applicable among all problems of a given type). When 
speaking in what follows about the evolution of theo- 
retical rationality, we will focus on the dimension of 
cognitive success. There are other, noncognitive aspects of 
human belief systems that are the target of evolutionary 
selection. One such aspect played a significant role in the 
evolution of religion and has been called the generalized 
placebo effect: the mere belief in a benevolent God (or sys- 
tem of Gods) equipped with supernatural powers, who 
protects one and rewards godly behavior (either earthly 
or afterworldly), has strongly positive effects on one’s 
psychological well-being, entirely independently of the 
cognitive implausibility of this belief.? Thus, although 
we focus our chapter on cognitive aspects, it should be 
clear that also certain noncognitive aspects play a role in 
the evolution of human belief systems. 

The goal of practical rationality is usually seen in 
guiding our actions in the interest of human beings. 
What this means depends on one’s accepted value judg- 
ments. As a minimal core meaning, we assume that act- 
ing in a practically rational way implies acting (a) in the 
egoistic interest of the acting person and (b) promoting 
cooperation between the interaction partners (possibly 
extended to nonhuman beings). Our discussion will 
focus on these two aspects of the evolution of practical 
rationality. 

The remainder of this article is structured as follows: 
after laying down the foundations of generalized evo- 
lution theory, including biological, cultural, and indi- 
vidual evolution (section 2), we discuss the interrelation 
between the evolutions of social cognition, language, 
and mental representation (section 3); the evolution 
of causal reasoning (section 4); the relation between 
modularity and universality in human cognition (sec- 
tion 5); and the evolution of induction and deduction 
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(section 6), and we conclude with a synopsis of the evo- 
lutionary architecture of human cognition (section 7). 


2. Cognitive Evolution in Three Dimensions 


Since our notion of rationality coincides with that of 
successful cognition, including social cognition, the 
focus of our chapter will be on the evolution of cogni- 
tion. Human cognition is the product of evolution in three 
dimensions. The first and second dimension are: 


(1) Biological evolution based on the biological inheritance 
of genetic information, including some recently 
explored epigenetic effects’ 

(2) Cultural evolution, where “culture” is intended in 
the broad sense of the generation-wise tradition of 
acquired information 


According to a more recent research paradigm, also the 
evolution of culture can be understood as a generalized 
Darwinian process, which is not based on the inheritance 
of genes through (sexual) reproduction but rather on the 
inheritance of cultural traits based on social learning. The 
latter notion means learning from other individuals, as 
opposed to individual learning from one’s own trial-and- 
error experience. Knowledge acquired through social 
learning (i.e., not inherited genetically) is passed on from 
generation to generation and is successively improved 
through ongoing iterations of variation-selection (or 
trial-and-error) cycles.* This is most impressively exem- 
plified in the evolution of technology from the tools of 
the Stone Age to contemporary automated industry and 
electronic intelligence (cf. Basalla, 1988), but it applies 
equally to other domains in the evolution of culture, such 
as science or law. 

Biological- as well as cultural-level evolution is based 
on the three Darwinian modules of reproduction, varia- 
tion, and selection (Dennett, 1995; Schurz, 2011; Sober, 
1993). In contrast, the third dimension of evolution is 
only partially Darwinian: 


(3) Individual learning, including inductive association 
learning and behavioral trial-and-error learning 


The latter kind of learning, also called “operant condi- 
tioning,” consists of Darwinian processes of variation 
and selection, insofar as spontaneously exhibited pat- 
terns of behavior are increased or decreased in their fre- 
quency, depending on whether the feedback from the 
environment was positive or negative for the individual. 
The only difference to a “full” evolutionary process is 
that individual trial-and-error learning is confined to the 
life span of an individual and cannot transcend it (except 
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when it becomes part of cultural evolution). Therefore, 
Schurz (2011, section 11.1.2) characterizes individual 
trial-and-error learning by means of Campbell’s notion 
of “retention” instead of reproduction (Campbell, 1960). 

In association learning, two stimuli frequently observed 
to be co-occurring are associated with each other in one’s 
mind and projected as expectation in regard to future 
observations. This is nothing but the inference of induc- 
tion as described by David Hume. A subcase of association 
learning is classical conditioning, as exemplified by Pavlov’s 
dog, which, due to a repeated ringing of a bell shortly prior 
to the giving of food, soon expects food already at hearing 
the bell ringing and starts to salivate. Classical condition- 
ing is a biologically widespread mechanism demonstrated 
almost everywhere in the animal world, even in worms 
(Bitterman, 2000; Delius, Jitsumori, & Siemann, 2000). 
Moreover, classical conditioning has a known neuro- 
logical basis, the so-called Hebb rule (Rojas, 1996, pp. 21, 
258-259). However, classical conditioning does not fol- 
low the Darwinian modules of spontaneous variation and 
selection but is a form of directly environment-driven (or 
“Lamarckian”) learning, since association hypotheses are 
directly “imprinted” upon the individual by the observed 
environment via inductive generalization. 

In conclusion, and in anticipation of the following, 
human cognition turns out to be based on the interac- 
tion of a large diversity of cognitive processes. Some of 
them are genetically determined, others are culturally 
acquired, and still others are individually learned. Even 
if a cognitive trait has a genetic basis, it does not usu- 
ally determine a person’s mind (thinking this would be 
so constitutes a frequent misunderstanding of genetic 
accounts). Because of the enormous plasticity of the 
human mind (cf. Churchland, 1979), there is presumably 
not even a single innate cognitive mechanism that can- 
not be overruled or corrected by other cognitive processes 
and in particular by culturally acquired abilities. For 
example, humans experience the Mueller—Lyer illusion as 
illustrated in figure 1.3.1: they see the left line as longer 
than the right line, although their length is equal. But 
the human mind is plastic enough to recognize this error, 
to correct it by reflective reasoning, and to recognize its 
cause, which lies in mechanisms of three-dimensional 
vision: the arrows and inverted arrows produce the three- 
dimensional effect that the right stroke appears to be 
closer to the observer than the left stroke and is thus per- 
ceived as being shorter (Rock, 1984, pp. 163, 167). 

As another example, consider Immanuel Kant’s famous 
thesis that perceived space is necessarily Euclidean and has 
three dimensions, because this is dictated by the innate 
(a priori) structure of human visual cognition. However, 
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Mueller-Lyer illusion. 


mathematically educated humans can overcome even 
this innate “barrier” and conceptualize non-Euclidean or 
more-than-three-dimensional spaces. Further examples 
will be given below. 

In the next four sections, we start with the most 
basic components of human cognition and proceed to 
increasingly higher forms. 


3. Social Cognition and the Evolution of Language 
and Mental Representation 


The most striking cognitive difference between humans 
and all other animals is the human faculty of language. 
Rudimentary abilities of symbolic signaling systems are 
already present in nonhuman animals, but while great 
apes can be trained to learn at most a few hundred words, 
humans can, in principle, produce a potentially infinite 
number of grammatical sentences from several hundreds 
of thousands of words and abstract syntax. According to 
a well-known theory advanced by philosophers (Gehlen, 
1956/1977; Topitsch, 1979), anthropologists (Boyer, 1994; 
Wenegrat, 1990), and biologists (Tomasello, 1999), the 
hominids’ language faculties have developed (presum- 
ably over 1-2 millions of years) in close interrelation with 
their social cognition, their abilities to communicate and 
to organize joint intention and cooperative action. One 
also speaks of a sociolinguistic coevolution (Heyes, 2012). 
Early hominid groups could even hunt animals as large 
as mammoths; a high amount of social coordination and 
division of labor is required for tasks of this sort. Although 
many aspects of human language appear to be cognitively 
universal, the initial selection pressure for the evolution 
of language was cultural. According to Thompson, Kirby, 
and Smith (2016), a strong genetic inheritance of gram- 
mar as postulated by Chomsky (1968) is not needed to 
explain the apparent universality of generative grammar 
in humans. By means of computer simulations and math- 
ematical proof, Thompson et al. (2016) demonstrate that 
a weak genetic bias in favor of a certain trait is sufficient 
to explain its universality in human cultures, since the 
genetic bias is amplified by learning processes. Jablonka 
and Lamb (2005) and Jablonka, Ginsburg, and Dor (2012) 
explain language evolution as a gene-cultural coevolution: 
various aspects of language initially invented culturally 
were later genetically assimilated (i.e., became partially 
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“inborn”) because they created genetic selection pressures 
for cognitive abilities specialized for learning language; 
this mechanism is also known as the “Baldwin effect” (cf. 
Dennett, 1995, pp. 77ff.). 

Communication—both about one’s intentions and 
about the environment—is an obvious presupposition 
of successful cooperation. The coordination of indi- 
vidual actions that is required for successful coopera- 
tive behavior is systematically analyzed in evolutionary 
game theory. Weibull (1995, pp. 28-31) classifies symmet- 
ric two-person games into three kinds: (1) Hawk—Dove 
games (possessing a stable mixed equilibrium); (2) coor- 
dination games (possessing several stable pure equilib- 
ria), including the famous stag-hunt game as a subcase; 
and (3) prisoner’s dilemma games (possessing no non- 
trivial equilibria). For the stag-hunt game, it has been 
shown (Skyrms, 2004) that under many but not all cir- 
cumstances, successful cooperative behavior can emerge 
but is not evolutionary stable: if too many individuals 
prefer hunting small prey (“hare”) instead of large prey 
(“stag”), because the success of the former does not 
depend on one’s partners’ cooperative behavior, then 
the evolution of cooperation breaks down. 

Even worse are the evolutionary chances of coopera- 
tion in the prisoner’s dilemma game, in which a cooper- 
ative action of helping the other is only optimal if it occurs 
reciprocally. Here cooperative actions are doomed to get 
exploited by “egoists” (i.e., noncooperators who receive 
help but don’t return help). Thus, in the prisoner’s 
dilemma game, cooperative action has to be stabilized 
by additional mechanisms of reciprocation.® Axelrod 
(1984/2006) proposed the strategy tit-for-tat (TFT), which 
initially acts cooperatively and then retaliates (i.e., pun- 
ishes defection with defection and rewards cooperation 
with cooperation). Axelrod suggested that TFT is uni- 
versally optimal and thus evolutionary stable, but later 
studies showed that there is no generally stable coop- 
eration strategy in the iterated prisoner’s dilemma, since 
for every refined cooperation strategy, one can invent a 
refined deception strategy that exploits it (Lorberbaum, 
1994). However, there are strategies by means of which 
the evolution of cooperation can at least be imperfectly 
stabilized, such as (1) group selection with ongoing 
regrouping (Sober & Wilson, 1998), (2) communication 
and correlated pairing (Skyrms, 2010), (3) institutional- 
ized sanctioning mechanisms (Giirerk, Irlenbusch, & 
Rockenbach, 2006), (4) cognitive mechanisms of cheater 
detection (Cosmides & Tooby, 1992), and (5) religious 
stabilization of altruism (Wilson, 2002). 

Historically, the strategy of reciprocation has first 
been rationally reflected in the earliest human juridical 
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document, the Babylonian Codex Hammurabi (1800 B.c.). 
But the evolution of cooperation begins much earlier. 
Many hunter-gatherer tribes were quite egalitarian, espe- 
cially in places where food supply was instable and food 
sharing important. In their comparatively small groups, 
egoistic behavior of group members was much more eas- 
ily controllable than in large agricultural societies (Sober 
& Wilson, 1998, p. 178; Wilson, 2002, p. 21). Thus, in all 
known hunter-gatherer societies, mechanisms of social 
reputation (“This guy is a good one”) have developed and 
are still effective in peer groups of contemporary societies. 
In contrast, in the larger communities of premodern states 
formed after the agrarian revolution around 10,000 years 
B.C., reputation mechanisms alone were not enough to 
prevent cooperation-undermining exploitation behavior, 
whence in all known agrarian proto-states, governmental 
institutions that sanctioned rule breakers and collected 
taxes (among other activities) have emerged. 

As explained, social cooperation requires the evo- 
lution of language and communication. But more is 
required for cooperation: the ability to represent and 
understand the minds, intentions, and beliefs of the 
other persons with whom one cooperates. Already at 
a very young age, human children develop the com- 
petence to understand other persons. At about nine 
months, babies acquire the ability of joint attention (i.e., 
they recognize the other’s gaze direction and can direct 
their own gaze to the object spotted by the other). Toma- 
sello (1999, pp. 61-70) speaks of the “nine-month revo- 
lution.” A few months later, human start to point toward 
objects of their interest, and at the same time, the learn- 
ing of language begins. Later, children acquire the ability 
of recognizing not only the others’ intentions but also 
their beliefs and of distinguishing these from their own 
beliefs. This capability of children has been called their 
ability of mindreading (Nichols & Stich, 2003), or their 
possession of a theory of mind (Leslie, 1987). It was first 
demonstrated in an experiment called the false-belief 
task (Wimmer & Perner, 1983). The result of this experi- 
ment was based on verbal instructions and supported 
the conclusion that the ability to understand (false) 
beliefs of others emerges in children at the age of about 
four years. However, more recent experiments using a 
violation-of-expectation method demonstrated that 
already 15-month-old infants seem to understand when 
a person has a false belief (Onishi & Baillargeon, 2005). 

There is a controversy about the nature of the cog- 
nitive mechanisms underlying human “mindreading” 
(cf. Nichols & Stich, 2003). While the theory-of-mind 
approach assumes a specialized cognitive module respon- 
sible for this faculty, the simulation approach argues for 
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the view that the same cognitive mechanism by which 
people represent their own intentions and beliefs is used 
for simulations of the (different) intentions and beliefs 
of other persons. An argument in favor of the simulation 
view is evolutionary simplicity: one cognitive system is 
evolutionary less costly and thus more probable than 
two separate systems (Schulz, 2011).’ 

At about six years, children acquire the conventional 
moral stage, according to the findings of Piaget (1932) 
and Kohlberg (1984). This stage goes beyond the mere 
understanding of the nature of reciprocity, which 
children already possess at the age of three or four (Cos- 
mides & Tooby, 2015, p. 653): it consists in the inter- 
personal generalization of reciprocal rules (if I have to 
follow the rule, you have to follow it, too), which estab- 
lishes a basic sense of normative justice together with 
the ability to detect rule-breakers. 

On a deeper level, the sociocognitive coevolution 
was not confined to communication and mindreading 
but went hand in hand with the evolution of higher- 
level systems of mental representation that intermediated 
between stimulus and response. A recent analysis of 
their evolutionary advantage has been given by Schulz 
(2018), thereby integrating and complementing related 
accounts of Millikan (2002), Papineau (2003), and 
Sterelny (2012). Schulz argues that connecting behavioral 
responses not directly with perceived stimuli, but rather 
with mental representations of object-configurations 
(which are in a separate step related to perceived stimuli), 
is cognitively much more efficient, because behaviorally 
relevant object-configurations (e.g., a bear in a cave) are 
not always indicated by the same perceptual cue but by 
different cues, dependent on the given circumstances. 
Note that in a similar way, the cognitive efficiency of the 
common-cause abduction of unobservable entities has 
been demonstrated in the philosophy of science (Schurz, 
2016). 


4. Evolution of Causal Reasoning 


Besides the sociolinguistic, there was a second coevo- 
lution, the sociotechnical coevolution (Heyes, 2012; 
Jablonka et al., 2012). Essential to it was the ability of 
hominids not only to predict but also to manipulate the 
objects in their environment, from fire usage, stone 
tools, weaving techniques, and advanced weapons to 
the invention of agriculture. Crucial for these develop- 
ments was humans’ capability to reason about the causes 
of (pleasant or unpleasant) events. Causal reasoning goes 
beyond mere inductive correlational reasoning: it reflects 
the causal structure behind correlations. For example, if 
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A is observed to be correlated with B, then according to 
widely accepted causal principles, (1) A may be the cause 
of B, or (2) B may be the cause of A, or (3) A and B may 
be effects of a common cause (which follows from the 
so-called causal Markov condition; cf. Pearl, 2009; Schurz 
& Gebharter, 2016; Spirtes, Glymour, & Scheines, 2000). 
Only in case (1) is it possible to change B by manipu- 
lating A. Thus, causal knowledge is crucial for humans’ 
ability to manipulate and control their environment. 

The earlier conjecture that causal reasoning is an exclu- 
sively human ability turned out to be untenable. Chim- 
panzees possess many systematic reasoning abilities, 
ranging from object classification and spatial movement 
to intelligent tool usage (Tomasello, 1999, pp. 16ff.). 
Also, other higher mammals have rudimentary causal 
reasoning abilities (Dunbar, 2000). However, humans’ 
capabilities of causal modeling are by far superior com- 
pared to their nonhuman relatives. Herrmann, Call, 
Hernandez-Lloreda, Hare, and Tomasello (2007) found 
that in simple causal tasks, great apes are about equiva- 
lent to three-year-old toddlers.* However, the big steps in 
causal and analytic reasoning in child development are 
achieved at ages between five and seven years (Brainerd, 
1978). The ability to mentally slip into the role or “mind” 
of someone else is the basis of pretend play in childhood 
(Leslie, 1987) and a crucial precondition for the child’s 
ability of counterfactual reasoning (Buchsbaum, Bridgers, 
Weisberg, & Gopnik, 2012; Rafetseder, Schwitalla, & Per- 
ner, 2013), which, according to philosophical analysis, 
constitutes an essential component of causal reasoning 
(Lewis, 1973; Woodward, 2003). 

According to the anthropological theory mentioned 
at the beginning of section 3, humans’ causal reason- 
ing evolved in close relation to their intentional reason- 
ing (i.e., their interpretation of the intentions of living 
beings as the causes of their behaviors). This hypothesis 
fits well with the cultural history of ideas. From the Stone 
Age to premodern civilizations, the major causal expla- 
nation mechanisms of Homo sapiens were twofold: some 
natural processes are causally explained by mechani- 
cal contact forces (e.g., throwing a stone or carrying a 
load), but natural processes such as the weather or the 
movement of celestial bodies, which emerge appar- 
ently unforced without an obvious mechanical cause, 
were explained by their animation according to the 
intentional model (Gehlen, 1956/1977). More specifi- 
cally, these natural processes were explained in terms 
of the same social-intentional model that explained the 
behavior of fellow men—a process that led to the poly- 
theistic conception of natural Gods (Topitsch, 1979; 
Wenegrat, 1990). 
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As rationality progressed in the cultural history of 
ideas, such explanations were rejected because of their 
lack of empirical support and replaced by explanations 
in terms of physical causes. However, the epistemic jus- 
tification of the assumption of causes governing the 
behavior of nonliving physical objects turned out to be 
a highly difficult problem in the history of philosophy. 
The idealist philosopher Leibniz attempted to reduce 
causal to generalized intentional notions. In contrast, 
the empiricist philosopher David Hume argued that 
observational evidence can give us only knowledge of 
correlations, not of causation. In contemporary philoso- 
phy, there are promising suggestions for solving Hume’s 
challenge (Schurz & Gebharter, 2016; Spirtes et al., 2000), 
but the controversy is still going on. 

The fact that human reasoning operates with two dis- 
tinct models of causality, physical versus intentional causal- 
ity, is confirmed by several psychological studies. Already 
at the age of six months, babies distinguish between non- 
living and living objects, to which they spontaneously 
apply these different paradigms of causality: while nonliv- 
ing objects can be put in motion only by the transmission 
of a force by physical contact, living objects are also able 
to move spontaneously without any external force, solely 
due to their intentions or internal causes. This finding was 
established by means of experiments in which babies were 
presented two sequences of events (cf. Leslie, 1982; Spelke, 
Phillips, & Woodward, 1995): in the “normal” sequence, a 
nonliving object (e.g., a ball) moves toward a second one, 
hits it, and pushes it away. In the “abnormal” sequence, 
the first object likewise moves toward the second one but 
already comes to a halt a distance away from it, and the 
second object moves away without touch “as if by magic.” 
While the babies hardly paid attention to the first event, 
their attention (measured by the duration of looking) 
lingers very long on the second event. In a second series, 
the same experiment was conducted with people instead 
of objects: a boy walks toward a girl; in the first case, he 
touches her and the girl walks away, and in the second 
case, he remains standing in front of the girl and the girl 
walks away after that. In this case, no significant differ- 
ence was found in the duration of the babies’ gaze: both 
sequences are equally “normal” for the babies. 

Thus, already babies distinguish spontaneously between 
contact causality for nonliving and intentional causality 
for living objects. In other words, intentional causality and 
contact causality are two distinct causal models. This also 
fits with the human history of ideas, insofar as contact cau- 
sality for inanimate objects not only is well anchored in 
intuitive human physics but has also dominated the his- 
tory of philosophy from Aristotle to Descartes. In contrast, 
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objects that moved without apparent contact causes (e.g., 
heavenly bodies) were consistently explained using the 
intentional model of causality as explained above. 


5. Modularity and Universality in Human Cognition 


A further debate concerns the generality versus specific- 
ity of human mechanisms of cognition. According to 
the school of evolutionary psychology (Barkow, Cos- 
mides, & Tooby, 1992), the human mind consists of 
various cognitive “modules.” These are cognitive pro- 
grams or processes that were evolutionarily selected for 
specific tasks (Carruthers & Chamberlain, 2000); they 
are genetically anchored and little mutable.’ Examples 
of such area- and purpose-specific modules have already 
been mentioned and include (1) the module of shared 
attention (developed in the child at 9 months), (2) the 
modules of intentionality and contact causality (at 6 
months), (3) the modules of language acquisition (from 
18 months on), (4) mental spatial models (at the sen- 
sorimotorical level with 0-2 years; Brainerd, 1978), (5) 
the theory-of-mind module (at 4-5 years), and (6) the 
development of conventional moral values (at 6 years), 
which are associated with (7) the module of cheater 
detection (see below). Related is the research program 
of adaptive rationality according to which humans’ cog- 
nitive mechanisms are adapted to the structure of local 
environments, being tailored to the specific tasks for 
which they provide highly efficient solutions (Gigeren- 
zer, Todd, & ABC Research Group, 1999; Todd, Gigeren- 
zer, & ABC Research Group, 2012). A second claim of 
proponents of this paradigm is that simple heuristics are 
frequently more successful than computationally costly 
general reasoning mechanisms, following the slogan 
“Less can be more.” 

The modularist view has also been called the “Swiss- 
Army-knife model” (Tooby & Cosmides, 1992) or the 
“adaptive toolbox model” (Gigerenzer & Selten, 2001). 
The opposite view is defended by generalist models 
of cognition. Until the 1960s, classical logic was the 
dominant generalistic normative standard of theo- 
ries of rational reasoning (cf. Inhelder & Piaget, 1958). 
When psychologists discovered empirically that, in 
many domains, human reasoning did not accord with 
the principles of logic (e.g., Wason, 1966), these find- 
ings were interpreted as signs of human irrationality (cf. 
Evans, 2002). It was suggested that psychologists should 
adopt an alternative general normative system, such as 
Bayesian probability and decision theory (e.g., Oaksford 
& Chater, 1991). However, human reasoning has been 
observed to deviate from the norms of probability, too 
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(Barbey & Sloman, 2007; Kahneman 2011; Kahneman 
& Tversky, 1972). These findings led many psychologists 
toward a general skepticism in regard to the “normative” 
status of generalist reasoning accounts. 

Contemporary defenders of generalistic reasoning 
accounts are more modest. They agree that the early 
models of reasoning in terms of logic or probability 
theory were too simplistic or unrealistic from a cogni- 
tive viewpoint. Moreover, they do not deny that human 
cognition hosts a multitude of specialized cognitive 
abilities or modules. However, it is implausible that the 
human brain contains a module for each specific pur- 
pose or area of application; 200,000 years of evolution 
are insufficient to allow so many modules (Tomasello, 
1999, pp. 204-205). Thus, contemporary generalists 
argue that besides specialized modules, cognitive evolu- 
tion has also brought about a set of purpose- and area- 
unspecific cognition mechanisms. This view has been 
defended, among others, by Over (2003, p. 122), Almor 
(2003, p. 104), and, more recently, in a volume on the 
“new thinking on the evolution of cognition” (Heyes & 
Frith, 2012). The authors in this book argue that human 
cognition is largely purpose-general; it does not resem- 
ble a Swiss army knife but rather a human hand, whose 
fingers can serve an extremely large variety of purposes 
(Heyes, 2012, p. 2092). 

Recently, the existence of generalistic “improvisa- 
tional intelligence” has also been acknowledged by 
some evolutionary psychologists (Barrett, Cosmides, & 
Tooby, 2007). For semantic clarification, note that the 
notion of a “cognitive module” in the mainstream lit- 
erature is always related to a specific type of adaptation 
problem, purpose, and environmental condition. Thus, 
one should not also call general cognitive abilities “mod- 
ules,” as this would stretch this notion over all bounds 
and make it empty. Moreover, it has been argued that 
purpose-specificity and domain-specificity are two dis- 
tinct dimensions of specificity/generality (Duchaine, 
Cosmides, & Tooby, 2001). However, it seems that the 
two—although clearly nonidentical—are definitely not 
independent: the number and diversity of purposes to 
which a cognitive ability is applicable and the number 
and diversity of conditions under which it is applicable 
are strongly correlated. 


6. Induction and Deduction 


The most important example of a general mechanism of 
learning and cognition is induction, which in the form of 
conditioning or association learning is common in vir- 
tually all animals (recall section 2). In the view of earlier 
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behavioral psychologists such as Thorndike (1911), the 
merit of learning by conditioning consists precisely in 
the fact that this is not an area-specific but a universally 
applicable learning process. 

General cognitive mechanisms are particularly impor- 
tant for sustainable evolutionary success under environ- 
mental conditions that are changing in unforeseeable 
ways—a situation that occurred frequently in hominid 
evolution and even more frequently in human cultural 
evolution. This argument has been put forward by Schurz 
and Thorn (2016) as an important complementation 
of the adaptive rationality account (see also Thorn & 
Schurz, 2019). To be cognitively successful under chang- 
ing environments, one needs strategies for learning 
which cognitive methods perform best in which envi- 
ronment, or temporal phase of the environment (a point 
acknowledged within the adaptive rationality program; 
cf. Todd et al., 2012, p. 15). Obviously, meta-cognitive 
selection strategies have to be sufficiently general; other- 
wise, they could not serve the purpose of selecting the 
optimal method in a wide range of environments. 

In view of the cognitive universality of inductive rea- 
soning, it is remarkable that the epistemic justification 
of induction turned out to be one of the most difficult 
problems in philosophy. According to David Hume’s 
famous challenge (1748/2006, chapters 4, 6), the reli- 
ability of inductive reasoning from past observations 
to future (unobserved) cases can in no way be demon- 
strated by rational argument, neither strictly nor even 
probabilistically. For this reasoning rests on the assump- 
tion that nature is uniform, that is, that the future 
resembles the past, but that this is so cannot be demon- 
strated by observation or by logic or probability calculus. 
Nor can it be demonstrated by induction, by pointing to 
the fact that so far, the method of induction was success- 
ful, because the “inductive justification of induction” is 
circular and thus epistemically worthless (for details, see 
Schurz, 2019, chapters 2-4). An alternative account of 
justifying induction, going back to Reichenbach (1935, 
§80), is based on the idea that induction is the best that 
we can do in order to achieve successful predictions. 
Schurz (2019) defends this idea in terms of the optimality 
of meta-induction. Given a type of problem (a prediction 
task or decision problem), the meta-inductivist observes 
the success records of all methods that are accessible to 
her (i.e., methods whose success rates can be observed). 
Using these records, the meta-inductivist constructs an 
optimal combination of these methods, in the form 
of a weighted average that is periodically updated in a 
success-dependent way. Based on discoveries in compu- 
tational learning theory (Cesa-Bianchi & Lugosi, 2006), 
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Schurz (2008, 2019) demonstrates that meta-induction 
achieves an optimal success rate in all possible worlds 
among all competing methods (of prediction or deci- 
sion) that are accessible to the given agent. Schurz 
(2008, 2019) proposes meta-induction as a new solu- 
tion to Hume’s problem of the justification of induction 
mentioned above. Since meta-induction is primarily a 
strategy of social learning—that is, learning from the 
performance of others (although it can also be applied 
to the results of individual learning)—its optimality con- 
stitutes an essential foundation for the social propaga- 
tion of information in cultural evolution (Schurz, 2012, 
2019, section 10.2). 

Induction and meta-induction are presumably the 
evolutionarily most fundamental cognitive mechanisms 
(insofar as all forms of learning from feedback are based 
on induction). They are certainly more fundamental than 
causal reasoning (see below) and even more fundamental 
than logical reasoning and deduction. On the other hand, 
from a philosophical viewpoint, logical reasoning is more 
fundamental than induction, since induction presup- 
poses some amount of logic and not vice versa. 

Psychologists have found out that human reasoning’s 
fit with the rules of classical propositional logic is weak: 
while some rules (e.g., modus ponens) are already mas- 
tered by children at the age of six (Brainerd, 1978), other 
logical rules such as modus tollens are not even under- 
stood by the majority of adults. The latter fact is exempli- 
fied in Wason’s famous card selection task (Evans, 1982, 
chapter 9; Wason, 1966). In this experiment (which is 
also discussed in chapter 2.3 by Johnson-Laird and chap- 
ter 4.5 by Chater & Oaksford, both in this handbook), 
test persons (TPs) are given four cards from a pack of 
cards and told to test the following rule: If there is an A 
on the front side, then there is a 1 on the back side. They see 
four cards lying on a table, two face-up, showing an A 
and a B, respectively, and two face-down, showing a 1 
and a 2. They were asked, Which of these four cards do 
you have to turn over in order to check whether the rule 
actually applies to these cards? According to the laws 
of classical propositional logic, one should turn over 
the first (A) card and the fourth (2) card: turning over 
the first card corresponds to the logically valid modus 
ponens (MP) inference (from “If A, then 1” and “A” infer 
“1”), Turning the fourth card over corresponds to the 
equally valid modus tollens (MT) inference (from “If A, 
then 1” and “not-1” follows “not-A”). In contrast, turn- 
ing over the second and third cards corresponds respec- 
tively to the two invalid inferences NA (“negating the 
antecedent”) and AC (“affirming the consequent”). The 
empirical performance results, however, are MP 100%, 
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NA 5%, AC 10%, and MT 5%. Thus, normal adults only 
master the logical rule of MP but fail to master the rule 
of MT and apply equally or even more often the invalid 
rules NA and AC. 

To defenders of the logico-generalist paradigm of cog- 
nition, these results came like a shock. Equally disturbing 
was a further result first brought to light by Griggs and Cox 
(1982), which demonstrated that people apply the rule of 
MT perfectly in the social context of detecting rule break- 
ers. Here, the TPs had to test the following consumption 
regulation in a youth club: Those who drink alcohol have to 
be at least 16 years old. TPs were confronted with four ado- 
lescents; of two of them (Berta, Klaus), they only knew the 
beverage but not the age (14 y., 18 y.), and of the other two 
(Lisa, Martin), they only knew the age but not the beverage 
(cola, beer). They were asked, Whom do you have to check to 
determine whether he/she has broken the rule? with the result 
that now all TPs checked Berta and Martin (i.e., they mas- 
tered this instantiation of MP and MT perfectly). 

As an explanation of this finding, Cosmides and 
Tooby (1992) proposed the thesis that humans have a 
specialized cognitive module for the detection of social 
rule breakers (cheater detection), which emerged through 
selection in the course of social evolution. More recent 
experiments (Cosmides, Barrett, & Tooby, 2010) show 
that these special inference abilities cannot be explained 
by assuming a more specific but still general reasoning 
ability, such as reasoning with deontic rules (as argued by 
Cheng & Holyoak, 1985) or with utilities (as proposed 
by Manktelow & Over, 1991); rather, they are restricted 
to situations in which the action (whose permission is 
governed by the rule) is beneficial to the actor, so that 
the rule violation is indeed an act of cheating. 

The fact that humans possess content-specific reason- 
ing modules does not exclude that their conditional rea- 
soning is also governed by some purpose-general rules. 
In this line, psychologists and philosophers have devel- 
oped the following evolutionary explanation for the 
deviations of human conditional (if-then) reasoning 
from the norms of deductive logic: the if-then relations 
of our natural environment are almost never strictly true 
but rather admit of exceptions (i.e., they express rela- 
tions of high conditional probability). The hypothesis of 
ordinary “If A then B” statements as high-conditional- 
probability statements (“B is highly probable given A”) 
was proposed by Adams (1975), evolutionarily substan- 
tiated by Schurz (2001), and has been experimentally 
confirmed (Evans, Handley, & Over, 2003; more on this 
in chapter 4.6 by Oberauer & Pessach, this handbook). 
It turns out that humans’ intuitive if-then reasoning fits 
well with the rules of probability logic (Pfeifer & Kleiter, 
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2010; Schurz & Thorn, 2012). The MT rule, however, is 
not probabilistically valid. This may explain why subjects 
rarely apply the rule MT in Wason’s experiment. In con- 
trast, in the second experiment, the more specific cogni- 
tive module of cheater detection becomes activated. In 
this way, both the specific module of cheater detection 
and the general reasoning mechanisms for uncertain 
conditionals have a plausible evolutionary explanation, 
without coming into conflict with each other. 

The interaction of specialized cognitive modules and 
general cognition mechanisms can also be excellently 
studied in the abovementioned example of the cognitive 
model of contact causality. Although already toddlers have 
the intuition that inanimate objects can be put in motion 
only through contact or pushing, people have known 
magnets for centuries. After a short time of adaptation to 
the seemingly “magical” actions at a distance of magnets, 
children just as adults become used to this without prob- 
lems. Apparently, the innate but, in the case of magnets, 
inapplicable model of contact forces is overwritten and thus 
corrected by inductive learning. We learn from this con- 
sideration that humans’ innate model of causality does 
not determine human reasoning by “a priori necessity” (as 
believed by the philosopher Immanuel Kant) but just acts 
as one innate disposition among many others and can be 
corrected by other cognitive mechanisms. 

Through examples like these, it becomes apparent 
how efficiently special and general cognitive mecha- 
nisms can work together. Over (2003, pp. 124-125) and 
Schurz and Thorn (2016) therefore propose a dualistic 
theory of cognition, according to which human cogni- 
tion consists of area-specific modules and general cogni- 
tive processes. A further example splendidly illustrating 
the division of labor between general and purpose- 
specific reasoning processes is the selection of optimal 
methods from the “adaptive toolbox” by the general 
strategy of meta-induction, as explained above. 


7. The Evolutionary Architecture of Human Cognition 


We have so far explained the cooperation of specialized 
(modular) and general mechanisms of human cogni- 
tion. Most modular cognitive processes, but also many 
general processes such as inductive inference, work for 
the most part unconsciously. What, then, is the actual 
role of consciousness in human cognition? There is no 
consensus on this question. An extreme view is epi- 
phenomenalism, which claims that consciousness only 
plays the role of a subsequently summarizing reporter 
on our unconscious mental processes but not the role 
of a causal trigger (Block, Flanagan, & Gtizeldere, 1996, 
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chapter 19). This view appears to me to be exagger- 
ated and even untenable. It is admittedly true that the 
conscious human mind reflects only a fraction of the 
unconscious processes of human cognition and often 
does play the role of a summarizing reporter. More- 
over, conscious human reasoning processes are slower 
by decimal powers than the unconscious and modu- 
lar cognitive processes. When we make practical deci- 
sions, for example, about whether we should quickly 
run across an intersection or not, then we gauge the 
involved probabilities and utilities intuitively within a 
fraction of a second, while if we calculated such an esti- 
mate in a precise decision-theoretic manner, it would 
take us hours. 

And still, this much-slower conscious, logical mind 
has proved in the course of cultural evolution that—if 
given enough time to pursue its activities in a secure 
space—it is capable of much greater cognitive achieve- 
ments than all cognitive heuristics put together. Its 
merits lie in its ability to comprehend its domain of 
application in a logically consistent and empirically 
controlled way, without any simplifications and dis- 
tortions. Through the institutional establishment of 
an area of research and education sealed off from the 
pressure of practical demands, the conscious mind was 
able to develop our scientific-technological civiliza- 
tion. Conscious reasoning, from Euclid and Leonardo to 
Minkowski, was able to systematize the mechanisms of 
Euclidean geometry and perspectival projection so com- 
pletely that all deceptions of our perception modules 
were resolvable. From Aristotle to Boole and Gédel, con- 
scious reasoning was able to capture the laws of logical 
and probabilistic inference so completely that thereby 
all intuitive errors of reasoning could be recognized. 
Through abstract-mathematical thinking, human rea- 
soning could penetrate areas exceeding everything that 
natural imagination and heuristics could ever give us: 
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Figure 1.3.2 
Evolution-based structure of human 
cognition. 


with Einstein (and others), it was able to intrude into the 
laws of the unimaginably large, with Bohr (and others) 
into the laws of the unimaginably small, with Darwin 
(and others) to transcend the border between the non- 
living and the living, and with computer technology 
to transcend the border between nature and mind. The 
overwhelming success of conscious scientific reasoning 
extended humans’ range of possible action and gave 
space to new technologies from synthetic chemistry and 
nuclear energy to genetic engineering and robotics. 

In conclusion, we can say that even if humans’ gen- 
eral reasoning capacities become efficient through their 
interaction with the specialized cognitive modules, 
it is still these general reasoning capacities that made 
scientific-technological progress possible and distin- 
guish humans most strongly from their nonhuman 
ancestors. This leads to modeling human cognition as a 
layered structure that is illustrated by the architectonics 
displayed in figure 1.3.2. 


Notes 


1. For the proof of this thesis of Hume by means of modern 
logic, see Schurz (1997). 


2. Religious beliefs in this generalized characterization are 
found in more or less all known cultures (including China), 
although there are important differences, for example, con- 
cerning polytheism versus monotheism or omnipotence versus 
power-restricted Gods or “ghosts” and so on (cf. Boyer, 1994; 
Schurz, 2011, section 17.5; Wenegrat, 1990). Another approach 
to the generalized placebo effect is the psychology of “positive 
thinking” (Taylor, 1989). 


3. Cf. Jablonka and Lamb (2005). These authors even postulate 


four dimensions of evolution. 


4. Pioneers of the modern theory of cultural evolution (to 
mention just a few) are Cavalli-Sforza and Feldman (1973), 
Dawkins (1976/1987), Boyd and Richerson (1985), and Donald 


110 


Campbell (1960); more recent developments are Mesoudi 
(2011), Schurz (2011), Dennett (2018), and Henrich (2016). 


5. Dawkins (1976/1987) suggested the name “meme” for the 
cultural counterpart to genes, but this notion is controversial 
(cf. Aunger, 2002). 


6. The notions of “egoistic” versus “cooperative” action are 
understood here in the evolutionary sense, not necessarily 
implying a corresponding psychological mechanism (cf. Sober 
& Wilson, 1998, p. 201). 


7. Tomasello (1999) has argued that the ability to conceive 
their conspecifics as intentional beings discriminates humans 
from nonhuman great apes. Later this thesis turned out to be 
untenably strong: also great apes possess these abilities but to 
a much lesser extent (Call & Tomasello, 2008). Based on the 
observation of eye movements, Krupenye, Kano, Hirata, Call, 
and Tomasello (2016) found evidence that chimps can even 
anticipate actions based on false beliefs. 


8. In contrast, human toddlers are better than chimpanzees in 
tasks of social cognition. 


9. Note that a genetically determined mechanism need not 
be “single track” but can be “multitrack,” leading to different 
behaviors in different environments (like a hardwired com- 
puter program can react differently to different inputs). In such 
a case, the phenotypic manifestation of the hardwired mecha- 
nism is mutable but not the mechanism itself. 
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1.4 Rationality and the Brain 
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Summary 


We are widely considered the “rational animal.” Coher- 
ence relations are essential for rationality. They allow 
for inferences that can identify actions consistent (or 
inconsistent) with achieving our goals, in the context of 
our beliefs. Consistent inferences can be further differ- 
entiated into those that are certain, plausible, and inde- 
terminate. Twenty-plus years of research on the neural 
basis of reasoning reveals no single unitary mechanism 
of inference. Rather, our system of reasoning can be 
understood in terms of multiple systems for generat- 
ing inferences and a common system for detecting and/ 
or resolving conflict or inconsistency. The systems for 
inference generation vary as a function of conceptual 
and logical relations. Within logical relations, they fur- 
ther vary as a function of argument form, presence or 
absence of belief-laden content, argument presentation 
modality, and determinacy of the conclusion. In this 
chapter, we organize and present the research on the 
neuroscience of reasoning along these lines. 


1. The Reasoning Animal 


Within the Western intellectual tradition, humans are 
widely regarded as the reasoning or rational animal. This 
is to say that our behaviors are explained by postulating 
beliefs and desires, as well as a principle of “coherence” 
that guides our pursuit of the latter in the context of the 
former. Rationality is instrumental: it is a means to an 
end. A rational choice is a deliberate choice or action 
(selected from a large/unbounded set) that moves an 
organism closer to its goals in a manner consistent with 
its knowledge and beliefs.’ For example, if I am thirsty 
and desire to drink water, and if I believe that there is 
a glass of water within reach on the right-hand side of 
my desk, and I reach out to the right-hand side, grasp 
it, bring it to my lips, and drink, my action is coherent 
or reasonable (in the context of my beliefs and desires). 
However, if, given the same desires and beliefs, I reach 


out to the left-hand side of my desk, that action would 
be incoherent or unreasonable because it is inconsistent 
with my belief that the glass is on the right-hand side 
and would not fulfill my desire to drink from it.” 

Iam using the term “coherence” as a basic, primitive, 
intuitive notion, meaning roughly “making sense.” It is 
a relationship between propositions. Consider the fol- 
lowing example: “If George is taller than Michael, then 
Michael is shorter than George.” You will recognize this 
statement as self-evident and true. But suppose that you 
refuse to accept its truth and ask me to prove it. What 
do I do? How can I possibly prove this to you? I can’t. It 
is like being asked to prove a postulate from Euclidean 
geometry. They are self-evident. Either you understand 
them or you don’t. But we all do understand Euclidean 
postulates and simple logical relations as self-evident. 
These basic, intuitive notions are enhanced and elabo- 
rated into sophisticated systems of reasoning, involving 
formal logic and probability theory, that we can learn 
and use, with varying degrees of success. 

Coherence relations can be broken down into seman- 
tic, logical, and conceptual relations. Semantic relations 
hold by virtue of the meaning of open-class words in 
the language. For example, a widow is a woman whose 
husband has died. Logical relations hold by virtue of the 
“closed-class” words in a language, such as “and,” “or,” 
‘Sfthen,” “all,” “some,” and “none,” and prepositional 
phrases, such as “greater than,” “inside of,” and so on. 
Each is associated with a fixed pattern of inference. Techni- 
cally, such inferences need not involve any knowledge of 
the world, only the language (but see below). Conceptual 
relations, on the other hand, involve evaluation of propo- 
sitions in light of our understanding of the world, includ- 
ing co-occurrence experiences and causal knowledge. For 
example, I may conclude that all dogs have tails, because 
all dogs that I have seen have had tails (co-occurrence), 
or I may conclude that the seasons are caused by tilting 
of the Earth on its axis, by having a (causal) model of the 
Earth’s orbit around the sun. We will confine our discus- 
sion to logical and conceptual relations. 
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Broadly speaking, the above types of inferences are 
our basic mechanisms for determining rational actions. 
They constitute a system for generating new beliefs from 
observations and/or existing beliefs and for maintaining 
consistency among our beliefs (i.e., our mental represen- 
tations of the world). They allow us to generate possible 
actions and identify those that are consistent or incon- 
sistent with achieving goals. Inconsistent actions can 
be ruled out. Consistent actions can be further broken 
down into those that are certain, plausible (but not cer- 
tain), and indeterminate. Any creature whose actions are 
a function of representations of the world—in particular, 
representations that have propositional content—will 
need some system of coherence maintenance to perform 
these dual functions of generating inferences (to guide 
actions) from perceptual input and existing beliefs, as 
well as maintaining the consistency of beliefs. 

We are creatures whose behavior is a function of our 
beliefs about the world rather than of the world itself. 
Beliefs are psychological attitudes toward mental repre- 
sentations that have propositional content. If I have the 
attitude of “belief” to the proposition “There is a tiger 
under my desk,” then I am asserting that a certain state 
of affairs is true of the world (namely, that there is a tiger 
under my desk). The source of this knowledge can be 
direct perception or inference based upon perception 
and/or other beliefs. 

Irrespective of source, to be useful and to facilitate 
survival and propagation, beliefs need to meet certain 
constraints: in the case of direct perception, they need 
to be veridical. In the case of inference, they need to 
be consistent. The issue of veridicality is largely self- 
evident. For instance, in the above tiger example, if 
my tiger beliefs are veridical with respect to the actual 
state of affairs, my engagement in tiger-avoiding behav- 
ior will be appropriate and conducive to my survival. If 
there is a mismatch between my beliefs and the facts in 
the world, my actions will be inappropriate. If there is 
no tiger under my desk but I believe there to be one, I 
will run away unnecessarily. If there is a tiger under my 
desk but I do not believe that there is one, I will be eaten. 
Beliefs that are not veridical are typically not useful (and 
may be harmful). Veridicality is a relationship between a 
representation and the world. Much perceptual and cog- 
nitive neural machinery is devoted to getting this rela- 
tionship largely right, most of the time. 

Apart from perception, inference constitutes the 
other important source of knowledge for humans. Infer- 
ences are drawn from perceptual input and/or existing 
belief networks. For example, suppose I observe 12 white 
swans on Lake Simcoe. I may be tempted to conclude, 
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“All swans are white.” This constitutes new knowledge, 
based on a conceptual inference from the observation 
of 12 swans. I then need to maintain this belief for it 
to guide my future behavior. There exist sophisticated 
long-term memory systems for this purpose. At some 
future date, I see a black swan at the zoo. This observa- 
tion is inconsistent with my previous belief “All swans 
are white.” If the inconsistency is detected, my belief 
“All swans are white” will need to be revised to “Most 
swans are white.” In the absence of this conflict detec- 
tion and revision, I would entertain the beliefs “All 
swans are white” and “All swans are not white” (because 
at least one swan is black). They cannot both be true of 
the world. 

The importance of the consistency of beliefs is not 
always appreciated but is as critical as the veridicality 
of beliefs. For example, if I hold the belief “Tigers are 
extremely dangerous” and also the belief “Tigers are not 
extremely dangerous,” what is it that I believe? More 
importantly, when confronted with a tiger, do I run away 
or do I ignore it and do nothing? Two different actions 
are mandated; one will lead to survival, the other to 
death. It is therefore not surprising that there should be 
considerable neural machinery devoted to maintaining 
consistency of beliefs. 

Therefore, a creature whose actions are a function 
of mental representations (with propositional content) 
needs to ensure that these representations meet the fol- 
lowing requirements: 


. Veridicality 
. Maintenance 
. Inference 


. Inconsistency detection 


nF WN 


. Updating/belief revision 


The veridicality requirement falls outside the scope 
of inference because, as noted above, veridicality is a 
relationship between propositions and the world. It is 
largely delegated to the perceptual systems. The main- 
tenance requirement ensures retention of beliefs and 
falls within the purview of memory systems. The system 
of inference is very much about generating new beliefs 
based on coherence relations between propositions. The 
inconsistency detection system ensures the detection of 
inconsistency among beliefs (which would be harmful 
to the organism). The belief revision requirement draws 
upon the inference and inconsistency detection appara- 
tus to maintain consistency among beliefs. 

The goal of this chapter is to summarize what we know 
about brain systems involved in inference generation and 


Rationality and the Brain 


consistency maintenance. In presenting this summary, 
I will not use cognitive theories of reasoning as organ- 
izing principles. Many articles and chapters organize 
data along one of the standard theories, be it mental 
models, mental logic, dual mechanisms, or a proba- 
bilistic account (Elqgayam & Over, 2012; Evans, 2003; 
Henle, 1962; Johnson-Laird, 1994). These theories were 
developed to account for behavioral data. They may pro- 
vide some insights for organizing the neuroscience data. 
However, making a priori wholesale commitments to 
one theory or another is probably counterproductive. 
My approach is to look at the neuroscience of reasoning 
data agnostically and to see what type of story might be 
embedded in it. 

It is for this reason that I have tried to step back and 
ask, “What is the evolutionary problem that our system 
of inference developed to solve?” The answer that I pro- 
posed above is inference generation and inconsistency 
detection in service of “truth preservation.” The neu- 
roscience data on reasoning suggest that the proximal 
mechanisms for solving these problems are multiple sys- 
tems for generating inferences and a separate common 
system for detecting and/or resolving inconsistency. 
The inference systems include a left prefrontal cortex 
(PFC) “interpreter” system that draws upon linguistic 
relations, a visuospatial system located in the parietal 
cortex that is engaged in linear comparisons, a visual- 
spatial system in the right PFC for set-inclusion deter- 
minations, and a system for dealing with indeterminate 
inferences, located in the right ventral lateral PFC. The 
generation systems do not guarantee consistency. A 
separate mechanism, located in the right lateral PFC, is 
provided for this purpose. It seems to be concerned with 
both the consistency between new information provided 
by the perceptual system and existing beliefs, as well as 
the consistency among propositional representations 
(internal or external). It is the latter issue that is relevant 
here.? 


2. Neurological Systems for Generating Conceptual 
and Logical Inferences 


The neuropsychological work on identifying systems for 
inference (or hypothesis generation) perhaps began with 
Sperry and Gazzaniga’s studies on split-brain patients 
and Gazzaniga’s conclusions regarding the dominance 
of the left hemisphere in generating inferences. In 
one classic experiment (Gazzaniga, 1989), split-brain 
patients were presented with a picture of a chicken claw 
projected to the right visual field (left hemisphere) and 
a picture of a snowy winter scene projected to the left 
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visual field (right hemisphere). The patient then had 
to select (one with each hand), from an array of other 
pictures, the two most closely related to the projected 
pictures. The patient selected a shovel with the left hand 
(because the right hemisphere, controlling that hand, 
had processed a snowy winter scene) and a chicken with 
the right hand (because the left hemisphere, controlling 
that hand, had processed a chicken claw). Upon being 
asked to explain the choice of the shovel with the left 
hand (guided by the right hemisphere), the patient’s left 
hemisphere (dominant for language) had no access to 
the information about the snowy scene processed by 
the right hemisphere. However, instead of responding 
“T don’t know,” the patient fabricated a plausible story, 
based upon background knowledge, and responded that 
the shovel was required to clean the chicken coop. Find- 
ings such as these led Gazzaniga (1998) to conclude that 
the left hemisphere was critical in drawing inferences. In 
fact, it couldn’t help itself. It seems compelled to com- 
plete patterns and impose order on an uncertain world. 
These initial findings have been fine-tuned over the 
years, and there is now considerable data to support 
the role of the left PFC in both semantic inference and 
simple logical inference. Unsurprisingly, however, the 
emerging picture is much more complex. The left PFC 
is not the only inference generation system in the brain. 
It is certainly involved in conceptual inferences. But its 
role in formal logical inferences seems much more con- 
strained. Formal logical inferences engage the bilateral 
PFC and the parietal cortex. The relative engagement of 
these two regions is a function of logical form and even 
presentation modality. Furthermore, within the same 
logical form, content effects and argument determinacy 
modulate the specific neural systems engaged. 


2.1 Conceptual Inference 

Conceptual relations involve evaluation of propositions 
in light of our understanding of the world, including co- 
occurrence experiences and causal knowledge. Consider 
the following example: 


(A) Eve is 42 years old. She is a serious and orderly 
woman. She loves a glass of good wine and playing 
chess. She tries to watch the news on foreign TV sta- 
tions every day. 


From this information, participants are much more 
likely to draw the inference “Eve is a librarian” than the 
inference “Eve likes to watch football.” Notice that nei- 
ther of these statements appear in the given informa- 
tion, nor follow logically from the given information. 
Nevertheless, the former inference is considered more 
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plausible than the latter, given what we have been told 
about Eve and what we know about the world in general 
(Tversky & Kahneman, 1974). 

Such content-based inferences are examples of induc- 
tive inferences. They draw upon our beliefs and knowl- 
edge about the world and tend to preferentially activate 
the left PFC. Goel, Gold, Kapur, and Houle (1997) and 
Goel and Dolan (2004) carried out inductive-inference 
studies with arguments such as in (B) and, with respect 
to the PFC, reported activation in the left dorsolateral 
PFC, Brodmann Areas (BA) 9, 8, and 45 (figure 1.4.1a). 


(B) Snakes are cold-blooded; 
Alligators are cold-blooded; 
.. All reptiles are cold-blooded. 


In a recent follow-up to these imaging studies, Goel, 
Marling, Raymont, Krueger, and Grafman (2019) had 
neurological patients with penetrating focal lesions 
engage in simple inductive inferences involving conclu- 
sions of variable believability, such as in the following 
arguments: 


(C) Rexdale is a German shepherd; 
Rexdale lives in Dtisseldorf; 


“. All German shepherds live in Diisseldorf. 


(D) Lipstick is moist and glossy; 
Fish scales are moist and glossy; 


. Lipstick is made from fish scales. 


(E) Snakes are reptiles; 
Snakes are cold-blooded; 


. All reptiles are cold-blooded. 


The conclusion of argument (C) is highly unbeliev- 
able because we know that German shepherds can be 
found in many cities. The conclusion of argument (E) 
(although technically false) is highly believable, given 
what we have been taught about reptiles. Most of us do 
not have strong beliefs about the manufacture of lip- 
stick and thus rate the believability of the conclusion 
in argument (D) as less certain. The authors report that 
patients with unilateral focal lesions to left BA (9 and 
10) have less intense beliefs about moderately believ- 
able (argument (D)) and highly believable (argument 
(E)) conclusions and are less likely to accept these argu- 
ments as plausible. 

Inductive reasoning has also been examined through 
the use of analogical mapping tasks. Wharton et al. (2000) 
had participants examine pictures of colored geometric 
shapes and determine whether the shapes were analo- 
gous (analogy condition) or identical (literal condition) 
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to a source picture of shapes. They reported enhanced 
brain activation in the medial frontal cortex (BA 8), the 
left PFC (BA 6, 10, 44, 45, 46, and 47), the anterior insula, 
and the left inferior parietal cortex (BA 40) when sub- 
jects made analogical-match judgments. 

Other studies have imaged brain activation associated 
with judgment of analogous word pairs as in example (F) 
(Green, Fugelsang, Kraemer, Shamosh, & Dunbar, 2006) 
and verbal analogies, as in example (G) (Luo et al., 2003). 


(F) Planet: sun versus electron : nucleus 


(G) Soldier is to army as drummer is to band. 


Green and colleagues (2006) found enhanced activation 
of parietal-frontal regions, most notably the left supe- 
rior frontal gyrus (BA 9 and 10) for word pair stimuli 
(example (F)). Examining analogous concepts (example 
(G)), Luo and colleagues (2003) reported activation in 
the bilateral PFC (BA 45, 47, and 11), the left temporal 
lobe (BA 22), and the hippocampus. 

Overall, studies evaluating language-based inductive 
arguments generally indicate activation in large areas of 
the brain, including the left frontal and parietal lobes. 
These regions overlap with the cortical regions involved 
in deductive reasoning with familiar material (discussed 
below). However, the evaluation of inductive arguments 
seems to be distinguished from the evaluation of deduc- 
tive arguments (such as arguments (H), (N), (O), (P) 
below) by greater involvement of the left middle frontal 
gyrus (BA 9) (Goel et al., 1997; Goel & Dolan, 2004). 


2.2 Logical Inference 

Logical relationships between propositions are set up 
by closed-class (logical) terms of language rather than 
open-class (content) terms. For example, categorical syl- 
logisms deal with quantification and negation, involv- 
ing reasoning with the words “all,” “some,” and “none,” 
as in arguments (H) and (I) below. Transitive arguments 
involve prepositional phrases such as “on top of,” 
” “inside of,” and 
“outside of” that can be used to build hierarchical rela- 


“shorter than,” “more expensive than, 


tions, as in arguments (J) and (K) below. Finally, studies 
involving sentential connectives are generally confined 
to conditionals and disjunctions, as in arguments (L) 
and (M) below (Eimontaite et al., 2018; Noveck, Goel, & 
Smith, 2004). Logical arguments are designed to be valid 
by virtue of their structure rather than content. In terms 
of formal logical inference, it doesn’t matter whether the 
arguments are about the color of broccoli or Julius Cae- 
sar crossing the Rubicon: they will be valid or invalid by 
virtue of their logical structure. Interestingly, this does 
matter to the brain. We return to this issue below. 
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Figure 1.4.1 

Systems for inference generation and conflict detection. (a-h) Systems for inference generation: (a) A left dorsolateral prefrontal cortex 
(PFC) system is involved in generating inductive inferences (reproduced with permission from Goel & Dolan, 2004); (b) a left 
lateral frontal-temporal linguistic system is activated during syllogistic reasoning involving content that we have beliefs about 
(reproduced with permission from Goel, Buchel, Frith, & Dolan, 2000); (c, d) bilateral frontal and parietal spatial systems are 
involved in formal syllogistic reasoning lacking meaningful semantic content (reproduced with permission from Goel et al., 2000); 
(e) linguistic transitive arguments with conclusions that we have beliefs about engage the left parahippocampal gyrus and the bilat- 
eral hippocampus (reproduced with permission from Goel, Makale, & Grafman, 2004); (f) linguistic transitive arguments involving 
conclusions that we have no beliefs about engage spatial systems in the bilateral parietal cortex (reproduced with permission from 
Goel, Makale, et al., 2004); (g) transitive arguments with pictorial stimuli engage the right rostral lateral PFC (reproduced with 
permission from Wendelken & Bunge, 2010); (h) indeterminate arguments with content that we have no beliefs about engage 
the right ventral lateral PFC (reproduced with permission from Goel, Stollstorff, Nakic, Knutson, & Grafman, 2009). (i) A common 
system for conflict detection: a common right lateral/dorsolateral PFC (BA 44/45) system seems to be engaged in detecting and/or 
resolving conflict or inconsistency (reproduced with permission from Goel & Dolan, 2003). 
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(H) All broccoli are vegetables; 
All vegetables are green; 
.. All broccoli are green. 


(I) All A are B; 
All B are C; 
.. ALLA are C. 


(J) London is north of Paris; 
Paris is north of Rome; 


.. London is north of Rome. 


(K) A is north of B; 
B is north of C; 
.. Ais north of C. 


(L) If it rains on Saturday, then Linda will not come to 
the barbecue; 


It is raining on Saturday; 


.. Linda will not come to the barbecue. 


(M) Tom went to the movies with either Linda or Mary; 
Tom did not go to the movies with Mary; 


.. Tom went to the movies with Linda. 


Types of logical argument forms Interestingly, different 
logical forms seem to call upon different neural machin- 
ery. A qualitative review by Goel (2007) and a quantitative 
meta-analysis by Prado, Chadha, and Booth (2011) identi- 
fied different brain systems engaged by categorical syllo- 
gisms, transitive arguments, and conditional arguments. 

For categorical syllogisms, as in examples (H) and 
(1), Prado et al. (2011) report activation in the left PFC 
(BA 9 and 44), the left precentral gyrus (BA 4), the right 
caudate, and the left putamen. For arguments involv- 
ing transitive relations, such as (J) and (K), they report 
activation in the bilateral parietal lobes (BA 7 and 40) 
and the bilateral dorsal PFC (BA 6). For conditional 
arguments, as in example (L), they report activation in 
a left hemisphere system involving the dorsal PFC (BA 
6) and the angular gyrus (BA 39). The results regarding 
categorical syllogisms and transitive inference are con- 
sistent with the qualitative summary provided by Goel 
(2007). The results for conditional arguments may be 
less robust due to the insufficient number of studies. As 
an example, a recent study by Baggio et al. (2016) reports 
activation in the left PFC (BA 44 and 47) for conditional 
reasoning. 

A number of individual neuroimaging and patient 
studies highlight the differential brain response to syl- 
logistic reasoning and transitive reasoning. Activation in 
the left lateral and dorsal PFC (BA 6, 44, and 45) is widely 
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reported for syllogistic reasoning tasks in neuroimaging 
studies (Goel, 2007; Goel et al., 2000; Goel & Dolan, 
2003; Reverberi et al., 2012). One of the few lesion stud- 
ies of deductive reasoning also reports that patients with 
left lateral and superior medial frontal lesions performed 
poorly on elementary deductive reasoning problems 
(Reverberi, Shallice, D’Agostini, Skrap, & Bonatti, 2009). 
Linguistically presented logical arguments involving 
transitive relations (examples (J) and (K)) activate the 
parietal cortex to a greater extent than the PFC (Goel, 
2007; Knauff, Fangmeier, Ruff, & Johnson-Laird, 2003; 
Modrono et al., 2018; Prado et al., 2011). In a patient 
study directly comparing reasoning in transitive argu- 
ments with categorical syllogisms, it was reported that 
patients with lesions to the parietal cortex were impaired 
in the former task but not the latter (Waechter, Goel, 
Raymont, Kruger, & Grafman, 2013). 

The meta-analysis study of logical form tells us some- 
thing important: there is no single mechanism for logi- 
cal inference. Examination of individual studies paints 
an even more nuanced picture. Generation systems vary 
as a function of content effects, type of spatial relations 
involved, argument determinacy, and even modality of 
argument presentation (linguistic versus pictorial). 


Content and logical form As indicated above, deductive 
arguments are valid or invalid by virtue of their logical 
structure. Logically, the semantic content of the premises 
does not make a difference to the validity of arguments. 
However, psychologically and neurologically, the seman- 
tic content of deductive arguments makes a significant 
difference. Given that we do not typically reason about 
As and Bs but about whether climate change causes hur- 
ricanes or whether one should buy a new or a used car, 
the issue of prior beliefs and semantic content of proposi- 
tions becomes a central one. 

Psychologically, the content effect is the finding that, 
despite deductive reasoning being a function of logical 
form, argument content modulates response. It is one of 
the oldest findings in the cognitive psychology literature. 
Wilkins (1928) noted that valid logical arguments with 
believable conclusions are much more likely to be rated as 
valid than valid arguments with unbelievable conclusions. 

A robust consequence of the content effect is the belief 
bias effect. In reasoning with content one has beliefs 
about, one will encounter either a congruency or an 
incongruency between the logical response and con- 
clusion believability. Congruent arguments are either 
valid with believable conclusions (argument (H)) or 
invalid with unbelievable conclusions (argument (N)). 
Incongruent arguments are either valid with unbelievable 
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conclusions (argument (O)) or invalid with believable 
conclusions (argument (P)). 


(N) No apples are fruit; 
All fruit contain calories; 


.. No apples contain calories. 


(O) All apples are fruit; 
All fruit are poisonous; 


. All apples are poisonous. 


(P) No apples are fruit; 
All fruit contain calories; 


.. All apples contain calories. 


Cognitive neuroscientists have examined content 
effects by comparing inferences involving meaningful 
contents (argument (H)) with inferences involving non- 
meaningful contents (argument (I)), as well as inferences 
involving different types of contents (arguments (H) and 
(J)). These studies allow us to determine not only the effect 
of content on neural processing but also the effects of dif- 
ferent types of contents. The neuroimaging data indicate 
that the main effect of comparing categorical syllogism 
arguments with meaningful semantic content (such as 
in (H)) with arguments without semantic content (such 
as in (I)) results in activation of a left frontal-temporal 
system, even after controlling for content (figure 1.4.1b) 
(Goel et al., 2000; Goel & Dolan, 2003). Comparing argu- 
ments without semantic content (such as (I)) with logi- 
cally equivalent arguments with semantic content (such 
as (H)) activates bilateral PFC (figure 1.4.1c,d), along with 
bilateral parietal and occipital regions. This result persists 
even after controlling for the presence or absence of con- 
tent (Goel et al., 2000; Goel & Dolan, 2003). When these 
comparisons are carried out with transitive arguments 
involving geographical knowledge, such as argument (J), 
and equivalent transitive arguments without semantic 
content, such as argument (K), the results show involve- 
ment of the left parahippocampal gyrus and the bilateral 
hippocampus in the meaningful-content condition and 
bilateral parietal involvement in the no-content condi- 
tion, even after controlling for content (Goel, Makale, & 
Grafman, 2004) (figure 1.4.1e,f). 

The involvement of the left frontal-temporal sys- 
tem in reasoning about familiar, meaningful content 
has also been demonstrated in neurological patients. 
In one study, Vartanian, Goel, Tierney, Huey, and Graf- 
man (2009) administered three-term transitive inference 
arguments to patients with frontotemporal dementia. 
The parietal lobes of these patients were largely spared. 
As predicted by the imaging studies, they performed 
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normally on arguments that they could have no beliefs 
about (such as (K)) but were selectively impaired in argu- 
ments involving meaningful content (such as (J)). 

In another patient study, Goel, Shuren, Sheesley, and 
Grafman (2004) administered the Wason card selection 
task to patients with focal lesions to either the left or the 
right PFC and found that all patients performed as well 
as normal controls on the arbitrary version of the task, 
but unlike the normal controls, patients failed to benefit 
from the presentation of familiar content in the mean- 
ingful version of the task. In fact, consistent with the 
neuroimaging data, the latter result was driven by the 
exceptionally poor performance of patients with lesions 
to the left PFC. Patients with lesions to the right PFC 
performed as well as normal controls. 

These results have been interpreted in the literature 
as the recruitment of different neural systems for con- 
tentful and noncontentful reasoning (Evans, 2003; Goel, 
2007). However, repetitive transcranial magnetic stimu- 
lation (rTMS) studies suggest an even finer-grained dis- 
tinction: Tsujii, Masuda, Akiyama, and Watanabe (2010) 
and Tsujii, Sakatani, Masuda, Akiyama, and Watanabe 
(2011) show that rTMS disruption of the left PFC spe- 
cifically reduces reasoning accuracy only on a subset of 
contentful reasoning trials, the congruent trials. If this 
is the case, it suggests that the left PFC’s role in logical 
inference may be limited to belief bias and conceptual 
connections and simple logical connectives. Arguments 
involving complex logical relations need to draw upon 
additional cognitive resources. 


Formal logical inference Logical arguments such as (I) 
and (Kk), which lack any meaningful semantic content 
that participants could have beliefs about, must be eval- 
uated with formal machinery (i.e., based purely on struc- 
ture). Arguments of the forms (H), (N), (O), and (P) result 
in engagement of the left PFC, while logically identical 
arguments lacking familiar content, as in argument (1), 
engage the bilateral lateral PFC along with the bilateral 
parietal and occipital lobes (Goel et al., 2000; Goel & 
Dolan, 2003) (figure 1.4.1c,d). 

One interpretation of these results would be that, while 
the left PFC may be necessary and sufficient to deal with 
logical inference involving familiar material that partici- 
pants have beliefs about (at least in congruent trials), the 
right PFC is part of the system required to deal with logi- 
cal inference in purely formal situations. Disruption of 
left PFC functioning would impair the content-sensitive 
inference system, resulting in poor performance on con- 
gruent trials. The rTMS data reported above support this 
prediction (Tsujii et al., 2010; Tsujii et al., 2011). 
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Stimuli presentation modality One seeming inconsis- 
tency in the neuropsychology-of-reasoning literature has 
to do with the neural basis of transitive inference. Above, 
we have reported that transitive inference involves largely 
the bilateral parietal cortex (Goel, 2007; Prado et al., 
2011). However, at least two studies (Fangmeier, Knauff, 
Ruff, & Sloutsky, 2006; Wendelken & Bunge, 2010) have 
focused on right BA 10 (medial anterior prefrontal and 
right rostrolateral PFC, respectively) as critical to “rela- 
tional integration” (figure 1.4.1g). Interestingly, these two 
studies differ from other studies of transitive reasoning in 
the neuroimaging literature by virtue of using nonlinguis- 
tic/pictorial stimuli. Waechter et al. (2013) propose that 
the modality difference across the studies (linguistic ver- 
sus pictorial) may account for the differences in results. In 
particular, when linguistic stimuli are used, greater effort 
and resources are required to map the stimuli onto spa- 
tial mental models as a prerequisite to solution (Johnson- 
Laird, 1983; Mani & Johnson-Laird, 1982). This requires 
the parietal cortex (Cohen et al., 1996; Goel & Dolan, 
2001; Knauff et al., 2003). In the case of pictorial stimuli, 
the spatial relations are actually exemplified or embodied 
in the stimuli; thus, this mapping has already been done 
in the task presentation, rendering the involvement of 
the parietal cortex less critical and perhaps shifting pro- 
cessing to the PFC. 


3. Indeterminacy Tolerance 


The purpose of inference is to generate new informa- 
tion (or at least make information that was previously 
implicit, explicit). For example, given the information in 
(Q), Ican make a determinate inference about the rela- 
tive population sizes of Toronto and Guelph—namely, 
that Toronto has a greater population than Guelph— 
without explicitly being told so. I can also be certain that 
the population of Guelph is not greater than the popula- 
tion of Toronto, because that would contradict the given 
information. 


(Q) The population of Toronto is greater than the popu- 
lation of Hamilton. The population of Hamilton is 
greater than the population of Guelph. 

(R) The population of Toronto is greater than the pop- 
ulation of Hamilton. The population of Toronto is 
greater than the population of Guelph. 


But what happens in the case of example (R)? Here 
we are explicitly told about the relative population sizes 
of Toronto and Hamilton, and of Toronto and Guelph. 
What inference can we draw about the relative popula- 
tion sizes of Guelph and Hamilton? In the absence of 
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any additional information, relying solely on the prem- 
ises, nothing follows. In this case, any conclusions we 
derive about the relative population sizes of Guelph and 
Hamilton will be invalid, not because of inconsistency 
with the premises but because of indeterminacy given 
the premises: there is no fact of the matter regarding the 
relative population sizes of Guelph and Hamilton, given 
the information provided in the premises. 

Goel et al. (2007) show that neurological patients with 
focal lesions to the left PFC have generalized impaired 
reasoning, including arguments with complete infor- 
mation (i.e., determinate, as in argument (Q)), while 
patients with right PFC lesions are selectively impaired 
only in arguments with incomplete information (i.e., 
indeterminate, as in argument (R)). This patient study 
demonstrates a double dissociation across left and right 
PFC along the dimension of determinacy. Neuroimag- 
ing studies involving similar transitive arguments reveal 
similar results (figure 1.4.1h) (Brzezicka et al., 2011; Goel 
et al., 2009). A study examining deductive reasoning 
with indeterminate syllogistic arguments also reported 
activation in the right PFC instead of the left PFC (Par- 
sons & Osherson, 2001). (A transcranial magnetic stimu- 
lation study involving spatial relational reasoning also 
showed involvement of the right superior parietal cor- 
tex in dealing with uncertainty, although it did not test 
for the involvement of the right PFC; Ragni, Franzmeier, 
Maier, & Knauff, 2016.) 

These data seem to suggest that we have developed 
special brain systems for dealing with indeterminate 
inferences. In cases where we can, we will fill in the miss- 
ing information, using the left hemisphere interpreter, 
but in cases where this is not possible, a right ventral- 
lateral PFC system is engaged to tolerate or accommo- 
date the indeterminacy. 


4. Detection of Conflict or Inconsistency 


Conflict or inconsistency can arise among one’s exist- 
ing beliefs, between existing beliefs and incoming new 
information, and from inferences drawn from existing 
beliefs or external propositions. Detecting and/or resolv- 
ing these conflicts or inconsistencies seems to be a gen- 
eralized function of the right lateral and dorsolateral 
PEC (figure 1.4.1i).* 

In the context of logical reasoning, we are specifically 
referring to an inconsistency between a response cued 
by our beliefs about the world and a response cued by 
the logical structure of the argument. Within incongru- 
ent trials (as in arguments (O) and (P)), the prepotent 
response is the incorrect response associated with the 


Rationality and the Brain 


believability of the conclusion. Incorrect responses in 
such trials indicate that subjects failed to detect and/ 
or overcome the conflict between their beliefs and 
the logical inference and/or to inhibit the prepotent 
response associated with the belief bias. These belief- 
biased responses activate the ventral-medial PFC (BA 
11 and 32), highlighting its role in nonlogical, belief- 
based responses (Goel & Dolan, 2003). The correct 
response indicates that subjects detected the conflict 
between their beliefs and the logical inference, inhib- 
ited the prepotent response associated with the belief 
bias, and engaged a formal reasoning mechanism. The 
detection of this conflict requires engagement of the 
right lateral and the dorsal-lateral PFC (BA 45 and 46) 
(see figure 1.4.11), while generating the logical response 
calls upon the visuospatial machinery in the parietal 
cortex (De Neys, Vartanian, & Goel, 2008; Goel et al., 
2000; Goel & Dolan, 2003; Prado & Noveck, 2007; Stolls- 
torff, Vartanian, & Goel, 2012). Knauff and colleagues 
(Hamburger et al., 2018; Knauff, 2013) make a related 
point in the context of visual imagery impeding spatial 
reasoning. 

These functional magnetic resonance imaging results 
have been replicated by rTMS studies demonstrating that 
stimulation of the right PFC specifically impairs perfor- 
mance on incongruent reasoning trials such as in argu- 
ments (O) and (P) (Tsujii et al., 2010; Tsujii et al., 2011). 
The rTMS data also show that disruption of the left PFC 
results not only in decreased performance in congruent 
trials but also in improved performance in incongruent 
trials. That is, when the left PFC is impaired, participants 
are less likely to go with the believability of the conclu- 
sion and will recruit other cortical regions to formally 
evaluate the argument. 

One early demonstration of this conflict detection 
system with lesion data was carried out by Caramazza, 
Gordon, Zurif, and DeLuca (1976) using simple two- 
term reasoning problems such as the following: “Mike is 
taller than George. Who is taller? Who is shorter?” They 
reported that left hemisphere patients were impaired 
in all forms of the problem but—consistent with imag- 
ing data (Goel et al., 2000; Goel & Dolan, 2003; Stolls- 
torff et al., 2012)—right hemisphere patients were only 
impaired when the form of the question was inconsis- 
tent with the premise (i.e., “Who is shorter?”). 

A final example of the role of the right PFC in conflict 
detection is provided by Reverberi, Lavaroni, Gigli, Skrap, 
and Bonatti (2005). They carried out a revised version 
of the Brixton task involving rule-induction and rule- 
conflict conditions. They reported that while patients 
with lesions to the left PFC showed an impairment in 
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rule induction, patients with lesions to the right PFC 
were impaired specifically in the rule-conflict condition. 
This conflict detection role of the right lateral and the 
dorsal PFC is a generalized phenomenon that has been 
documented in a wide range of paradigms in the cog- 
nitive neuroscience literature (Fink et al., 1999; Picton, 
Stuss, Shallice, Alexander, & Gillingham, 2006; Vallesi, 
Mussoni et al., 2007; Vallesi, Shallice, & Walsh, 2007). 
Marinsek, Turner, Gazzaniga, and Miller (2014) have actu- 
ally suggested that conflict or inconsistency detection is 
the main role of the right PFC in cognitive functioning. 


5. Summary and Conclusion 


Humans are creatures whose behavior is a function 
of their beliefs about the world rather than the world 
itself. Not only do we have beliefs, but our beliefs also 
have propositional content. Beliefs with propositional 
content allow us to generate new knowledge by draw- 
ing inferences that take us beyond direct perception and 
differentiate between what is necessarily the case, what 
might be the case, and what absolutely cannot be the 
case. One can even time-travel with such mental repre- 
sentations and entertain past and future possibilities, 
including counterfactuals. 

For such a system to be useful, the inferences must 
be coherent, and this coherence must be maintained 
over the whole system of beliefs. There must also be a 
system for detecting inconsistencies in inferences and 
among beliefs. Twenty-plus years of neuroscience stud- 
ies of logical reasoning have revealed that there is no 
unitary reasoning module in the brain for undertaking 
this. Rather, our ability to reason seems to be underwrit- 
ten by two separate classes of mechanisms: (1) mecha- 
nisms for hypothesis generation and inference and (2) a 
mechanism for detecting conflict or inconsistency. 

Inference generation calls upon several different sys- 
tems, including (1) a left PFC interpreter system sensitive 
to semantic, conceptual, and simple logical relations; 
(2) multiple visuospatial systems; and (3) a system for 
tolerating indeterminacy. The first of these systems 
deals largely with semantic and conceptual relations and 
simple syntactic inferences. The second deals with more 
formal processing of logical arguments. The neurologi- 
cal basis of this system seems to vary as a function of 
logical form and argument presentation modality. For 
example, set-inclusion relationships call upon the right 
and/or bilateral PFC, while linear comparisons call upon 
the parietal cortex. Additionally, linear comparisons pre- 
sented pictorially activate the right rostral PFC, while 
the same linear comparisons presented linguistically 
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activate bilateral parietal cortex. Finally, a system in the 
right ventral-lateral PFC seems to play a critical role in 
allowing for indeterminate inferences. 

These systems of inference generation do not, in and 
of themselves, guarantee consistency. We seem to have 
a separate system in the right lateral/dorsolateral PFC 
for detecting conflict or inconsistency between external 
and internal representations and among internal repre- 
sentations. Together, these various systems account for 
the ability to draw correct inferences and maintain the 
consistency of the overall belief network. 

The details of this overall account will undoubtedly 
change as additional studies are carried out and new 
data are generated and added to our knowledge base. 
However, after 20 years of research, the broader picture 
that is emerging may be reasonably secure: specifically, 
that there is no single system of logical reasoning in the 
brain. Our ability to engage in rational thought is under- 
written by several different types of inference systems 
and a common system for detecting inconsistencies. 


Notes 


1. Additionally, while not germane to our purposes here, it is 
important to note that there is a “gap” between input and output 
conditions in rational actions. The antecedent condition is never 
causally sufficient for the consequent condition (Cassirer, 1944). 


2. Sometimes a distinction is made between “instrumental” 
rationality and “epistemic” rationality (Stanovich, 1999). The 
latter is an evaluation of the fit between an individual’s beliefs 
and the facts in the world. However, this is simply instrumen- 
tal rationality applied to belief evaluation and revision. It is not 
clear that this distinction needs to be dealt with separately, at 
least for our purposes. 


3. Most neuroscience discussions of rationality usually begin 
and end with delusions. While delusions are an important 
topic, I will not touch upon them here. I think we will increase 
our chances of understanding delusions if we can first under- 
stand the neural basis of normal rationality. 


4. Experimental data have thus far not clearly distinguished 
between the right PFC’s role in detecting and resolving the 
conflict. 
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Section 2 Key Issues of Rationality 


2.1 Reasons and Rationality 


John Broome 


Summary 


I explore the relationship between rationality and rea- 
sons, particularly the reductive idea that rationality can 
be defined in terms of reasons. I start with an analysis 
of the meaning of “rationality” in order to clarify the 
issue. Then I assess the view that rationality consists in 
responding correctly to reasons. To this I oppose a “quick 
objection,” describe the defenses the view has against 
this objection, and argue that these defenses are unap- 
pealing. Next, I assess various related views, including 
the view that rationality consists in responding correctly 
to beliefs about reasons and argue against each of them. 
Eventually, I identify the kernel of truth that lies within 
them, which is that rationality requires you to intend 
to F if you believe you ought to F. I call this principle 
“enkrasia.” It is only one requirement of rationality 
among many, so it licenses no reduction of rationality. 


1. Normativity and Reasons 


Knauff and Spohn explain in their Introduction that 
a recurring theme in this handbook is the relation 
between positive and normative approaches to the study 
of rationality. This chapter investigates some fundamen- 
tal aspects of the normative approach. 

The word “normative” has various meanings, and 
at least two are current in philosophy. One meaning is 
“involving correctness.” Rules and requirements are by 
definition normative in this sense. Any rule or require- 
ment sets up a standard of correctness, so that complying 
with it is correct according to the rule or requirement. 
Rationality requires things of us, or—to put it differ- 
ently—it prescribes things to us. For instance, it pre- 
scribes that we intend means to ends that we intend and 
that we do not have contradictory beliefs. So rational- 
ity is inevitably normative in this sense of “normative.” 
Much of the study of rationality is concerned with its 
normativity in this sense, investigating just what ratio- 
nality requires of us. 


Many things besides rationality are normative in this 
sense. Fashion is an example. Fashion these days pre- 
scribes that men do not wear bellbottom trousers. But 
this does not automatically imply that a man ought not 
to wear bellbottom trousers or has any reason not to. It 
is a real question whether we have any reason to dress as 
fashion requires. This a question about the normativity 
of fashion in a different sense. In this sense, “normative” 
means “involving ought or reasons.” 

We may ask the same question about rationality: when 
rationality requires something of us—such as to intend 
means to ends we intend—does that imply we ought to 
do it or have any reason to do it? Although rationality is 
inevitably normative in the first sense, there are real ques- 
tions about its normativity in the second sense. These 
and related questions are the topic of this chapter. In this 
chapter, “normative” has the second sense. 

Reasons are a paradigmatic feature of normativity in 
this sense. They became an important object of study for 
philosophers only in the middle of the 20th century. A 
significant achievement of the philosophy of normativ- 
ity since that time has been to make a sharp distinction 
between motivating reasons and normative reasons (e.g., 
Smith, 1994). (To be precise, the distinction is between 
the property of being a motivating reason and the prop- 
erty of being a normative reason. Many particular things 
have both properties.) Motivating reasons explain, or 
help to explain, why a person does something. Normative 
reasons explain, or help to explain, why a person ought 
to do something (Broome, 2013, chapter 4). It is norma- 
tive reasons that figure in this chapter, since the chapter 
is about the relation between rationality and normativity. 

In the past few decades, reasons have come to domi- 
nate the philosophy of normativity. As a result of what 
is often called “the reasons-first movement,”' many 
philosophers now think that rationality can be given 
a reductive definition in terms of reasons. If that were 
true, it would mean that the study of rationality is noth- 
ing more than the study of reasons. One aim of this 
chapter is to explore this reductive idea. 
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2. The Meaning of “Rationality” 


It is natural to associate reasons with rationality. The 
words “reason” and “rationality” have a common origin 
in the Latin word “ratio.” But this simple etymological 
association covers up a tangle of meanings that connect 
the two words. I need to start with some disentanglement. 

The word “reason” entered English from French 
along with the Norman invasion of England in 1066. Its 
first recorded occurrence in English is in a book called 
the Ancrene Riwle, whose earliest manuscript dates from 
about 1225 (Day, 1952). “Reason” appears there in vari- 
ous different senses, all of which survive today. Some- 
times it means simply “explanation,” as it still does in 
such sentences as “The reason for the long delay was 
incompetence.” Often it refers to a motivating reason, 
which is a special sort of explanation of why a person 
does something. 

Just once in the Ancrene Riwle, “treason” refers to a 
normative reason. This is in the sentence (translated 
into modern English): 


The third reason for fleeing the world is the gaining of 
heaven. (p. 73, folio 43) 


Just previously in the text, the author says he will describe 
“eight reasons why one ought to flee the world,” which 
is to say, eight explanations of why one ought to flee the 
world. Then he starts to enumerate them, and when he 
comes to the third, he describes it as a reason for fleeing 
the world. So “a reason for fleeing the world” refers to an 
explanation of why one ought to flee the world. This is a 
normative reason.” 

In all those senses, “reason” is a count noun. It also 
appears once in the Ancrene Riwle as a non-count noun 
naming a property that people possess. We still call this 
property the “faculty of reason.” Since it is a mental fac- 
ulty, let us call this “reason in the mental sense.” The 
original text needs some exegesis: 


Wummon is the reisun—thet is, wittes skile—hwen hit 
unstrengeth. (p. 121, folio 73) 


The author has just recounted a parable from the 
Bible. He is saying that the woman in the parable repre- 
sents the faculty of reason (spelled “reisun”). Because the 
word “reason” had only recently acquired the mental 
sense, he glosses it using an older English term for the 
faculty of reason: he says (in modern spelling), “that is, 
wit’s skill.” Since this earliest mention of the faculty of 
reason is obscure, here is a clearer one from Shakespeare 
(A Midsummer Night’s Dream, Act 2, Scene 2): 


The will of man is by his reason sway’d. 
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The adjective “rational” is first recorded by the Oxford 
English Dictionary (OED) in 1398. From its beginning, it 
was cognate to “reason” in the mental sense and in that 
sense only. It had the meaning “having the faculty of 
reason.” It had this meaning and no other for about two 
hundred years. The OED shows that for all that period, 
it was applied as a predicate only to people, creatures, 
souls, minds, and suchlike: all things that could possess 
the faculty of reason. This meaning of “rational” persists 
today. 

The noun “rationality” appeared in 1627 as the name 
of the property that is ascribed by this adjective. Since this 
property is just reason in the mental sense, “rationality” 
and “reason” in this sense were originally synonyms. 

However, the meaning of “rationality” has by now 
broadened. “Reason” in the mental sense refers only to a 
faculty. “Rationality” today refers to the same faculty and 
also to a state of mind—roughly, a state of mind that could 
have arisen from the exercise of the faculty of reason, 
which is to say, a coherent state of mind. The term “struc- 
tural rationality” is often used today for the rationality of 
states.’ These days, we would not count a person as fully 
rational if she had the faculty of rationality but not struc- 
tural rationality. For instance, a person is not fully rational 
if she does not intend means to her ends, even if she has 
the ability to ensure that she does intend means to her 
ends. Ability is not enough; we expect it to be exercised. 

Nevertheless, even in this broadened sense, rational- 
ity retains one central feature: it is a property of a per- 
son and specifically a property of her mind. Moreover, 
it depends on the other properties of the person’s mind: 
as Wedgwood (2002) puts it, rationality supervenes on— 
depends only on—the mind. If a person might be in 
either of two possible situations, but her mental proper- 
ties apart from rationality would be the same in either, 
she would be equally as rational in one as in the other. 
So, even though the meaning of “rationality” has broad- 
ened beyond the mental sense of “reason,” it still refers 
to a mental property. 

However, from 1598 onward, the OED records “ratio- 
nal” used as a predicate of things that do not have 
minds. These days, we apply “rational” to acts, beliefs, 
city plans, and many other things without minds. These 
uses of “rational” for nonmental things are derived from 
the original, mental sense applied to people. A city plan 
is rational if it could have been designed by rational 
people. A person’s act is rational if, were she to do it, she 
would be no less rational than if she were not to do it. 
And so on. Nevertheless, in its core meaning, “rational- 
ity” still refers to a property of people, and it is a mental 
property. It supervenes on the mind. 
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Some philosophers assign a different meaning to “ratio- 
nal.” For example, Kolodny and Brunero (2013/2018) say, 


“What would it be rational for an agent to do or intend?” 
could mean: 


1. By doing or intending what would the agent make her 
responses (i.e., her attitudes and actions) cohere with one 
another? ... 


2. What does the agent have reason, or ought she, to do or 
intend? 


The first of these meanings is roughly structural rational- 
ity. The second is sometimes called “substantive ratio- 
nality,”* but it is not a normal meaning of “rational” at 
all. To see this, think of a case where you ought to turn 
left, but you firmly believe on the basis of strong but 
misleading evidence that you ought to turn right. In the 
substantive sense, it would be rational for you to turn 
left. But no nonphilosopher would say it is rational. No 
one would call it rational to do the opposite of what you 
firmly believe you ought to do.° 

The substantive meaning of “rational” could be ety- 
mologically justified. “Rational” is a cognate word to 
“reason,” and it could in principle be cognate to “rea- 
son” in the normative sense. Actually, however, “ratio- 
nal” has never had this normative meaning in common 
English. It has always been cognate to “reason” in the 
mental sense only. I use it only with this common 
meaning. 


3. Rationality as Responding Correctly to Reasons 


Although “rationality” does not have a normative mean- 
ing, it is a popular view among philosophers and others 
that, as a substantive matter, rationality is nevertheless inti- 
mately connected with normativity. A strong version of 
this view is the claim that rationality consists in respond- 
ing correctly to reasons.° This is a reductive claim: it claims 
that rationality is reducible to reasons in this way. 

It is subject to something I call the “quick objection. 
The property of rationality supervenes on the mind, 
whereas the property of responding correctly to reasons 
does not. These therefore cannot be the same property. 
This section examines the quick objection and defenses 
against it. 

Take a particular person called “you.” You are ratio- 
nal to some degree, and this degree supervenes on your 
mind. I insisted in section 2 that this is part of the mean- 
ing of “rational.” 

What about responding correctly to reasons? First, 
what is this property, more exactly? You have many rea- 
sons. Each is a reason for some particular thing: a reason 
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for you to do something, or to believe something, or not 
to intend some particular end without also intending 
a means to it, or for something else. I use a schematic 
letter to represent this generality: a reason of yours is 
a reason for you to F. Responding correctly to reasons 
cannot be simply Fing whenever you have a reason to F. 
Often you have a reason to F and also a reason not to F. 
You cannot both F and not F, so if responding correctly 
to reasons required you to F whenever you have a reason 
to F, you often could not respond correctly to reasons. 
That cannot be so. 

Instead, we must recognize that your reasons in some 
way combine together. They may weigh against each 
other, some may override others, some may cancel others, 
and so on. Your reasons together require various things of 
you. They require you to F, to G, and so on. Another way 
of putting this is that you ought to F, to G, and so on. 

In this section, I assume that, to respond correctly 
to reasons, you must comply with reasons, by which 
I mean you must F whenever your reasons together 
require you to F.® An alternative interpretation is that 
you must intend to F whenever your reasons together 
require you to F. I consider that interpretation in sec- 
tion 5. Your reasons together could not both require you 
to F and require you not to F, so the previous problem 
does not arise for either interpretation. 

Responding correctly to reasons may imply not just 
complying with reasons but also doing so because your 
reasons require you to. In this section, I assume that 
complying with reasons is at least a part of respond- 
ing correctly to reasons. Consequently, the property of 
responding correctly to reasons cannot supervene on 
your mind unless the property of complying with rea- 
sons does. I shall argue that complying with reasons 
supervenes on your mind only if some unappealing 
philosophical theories are true. 

For you to comply with reasons is for the following 
universal conditional proposition to be true: that, for 
any F, you F if reasons require you to F. This conditional 
supervenes on your mind if both sides of it do—that is, 
if, for any F, first, whether or not reasons require you 
to F supervenes on your mind and, second, if reasons 
require you to F, whether or not you F supervenes on 
your mind. As I shall put it: first, what your reasons 
require supervenes on your mind and, second, your 
performance supervenes on your mind. It is conceiv- 
able that the property of complying with reasons could 
supervene on your mind even if one of these conditions 
was not satisfied, but I cannot see how this could actu- 
ally happen. So the quick objection divides into two 
objections. The first is that what your reasons require 
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does not supervene on your mind. The second is that 
your performance does not supervene on your mind. 
Either is enough to refute the claim that rationality con- 
sists in responding correctly to reasons. I shall develop 
these two objections in turn. 


3.1 First Objection 

Does what your reasons require of you—in other words, 
what you ought—supervene on your mind? The claim 
that it does is a sort of subjectivism about ought. Vari- 
ous subjectivist theories support it. For example, one is 
the theory that you ought to F if and only if Fing has 
the greatest expected value for you out of all the alter- 
natives, where expected values are given by your own 
credences and your own judgments of value. 

Many philosophers find subjectivism about ought 
an unappealing theory. It conflicts with common sense, 
if nothing else. Common sense tells us that external 
facts can influence what you ought to believe or do. For 
example, the fact that lowering clouds are gathering is 
a reason to expect rain, and the fact that your child is 
badly hurt is a reason to take her to the hospital. 

Kiesewetter (2017, chapter 7) offers a means of easing 
this discomfort with subjectivism. He agrees with com- 
mon sense that reasons are often features of the external 
world and argues that this can be made consistent with 
subjectivism about ought. 

His argument is this. A feature of the external world 
is a reason for you only if it available to you, by which 
he means it is part of your body of evidence. Indeed, 
he assumes that what you ought is determined by your 
total body of evidence together with features of your 
mind such as your likes and dislikes. He now applies a 
strong dose of externalism about the mind, taking his 
lead from Williamson (2000). According to Williamson, 
your evidence is what you know, and your knowledge 
is a mental state of yours. Given this, your body of evi- 
dence is a feature of your mind. So what you ought 
is entirely determined by features of your mind, even 
though reasons are features of the external world. 

I doubt this will ease many philosophers’ discom- 
fort.’ It conflicts equally with common sense. In effect, 
it expands the notion of the mind to include whatever 
facts in the world constitute reasons. Subjectivism about 
ought remains an unappealing theory. 


3.2 Second Objection 

Your reasons often require you to act in the external 
world. For example, your reasons may require you to 
insure your house. Even reasons that are features of 
your own mind may require this. For example, perhaps 
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you want to avoid risk of financial ruin and believe that 
insuring your house is necessary for that purpose, and 
perhaps this desire and belief constitute a reason for you 
to insure your house. 

So let us assume your reasons require you to insure 
your house. Suppose you take the usual steps to do so: 
you complete an application form, glance through the 
contract, pay the premium, and so on. Compare two 
cases. In the first, by these steps, you successfully insure 
your house. In the other, a clause in the contract, which 
you do not read, says your house is insured only if it is 
roofed with metal, tiles, or slate. Your house is roofed 
with cedar shingles, so you do not successfully insure 
it. But suppose you never claim on insurance, and your 
failure never comes to light. Then your mind has all the 
same properties in both cases. Nevertheless, in one you 
do as your reasons require and in the other you do not. 
So your performance does not supervene on your mind. 

This second objection could be overcome if we could 
accept a sort of subjectivism about performance. We 
could say that reasons cannot require you to do some- 
thing unless the criterion for whether or not you do it 
is internal to your mind. We could deny in the example 
that your reasons require you to insure your house. We 
could say instead that they require you to act in a way 
that appears to you to be insuring your house or, alter- 
natively, that they require you to intend to insure your 
house. 

This sort of subjectivism has been defended,"° but it, 
too, is unappealing. The relevant reason in this case is a 
reason of self-interest: it is in your interest to insure your 
house. It is not in your interest to do something that 
appears to you to be insuring your house or to intend to 
insure your house, except insofar as either leads you to 
actually insuring it. 


3.3. Conclusion 

The claim that rationality consists in responding cor- 
rectly to reasons can be defended against the quick 
objection only by showing that complying with reasons 
supervenes on your mind. There are two objections to 
this claim, which can be overcome only on the basis of 
unappealing philosophical theories. The quick objection 
is vindicated to this extent. 

In any case, blocking the quick objection is far 
from sufficient to establish that rationality consists in 
responding correctly to reasons. There are other, inde- 
pendent objections. One is that it is often moral reasons 
that require you to do some act. Suppose you respond 
correctly to these reasons by doing this act because your 
moral reasons require you to. If responding correctly to 


Reasons and Rationality 


reasons constituted rationality, this would exhibit your 
rationality. But actually, it exhibits your morality rather 
than your rationality (see chapter 12.1 by Fehige & Wes- 
sels, this handbook). 

The same would be true even if your responding to 
reasons supervened on your mind. Suppose that your 
moral reasons require you not to have racist beliefs or 
not to have evil intentions, for example. Again, respond- 
ing correctly to these reasons exhibits your morality and 
not your rationality. 

The claim that rationality consists in responding cor- 
rectly to reasons remains dubious. 


4. Rationality as Entailed by Responding Correctly 
to Reasons 


A weaker claim is that rationality is entailed by respond- 
ing correctly to reasons.'' This, too, may be intended as 
a reductive claim that rationality is nothing more than a 
part of responding correctly to reasons. 


4.1 Structural Reasons 

On one version of this view, everyone has reasons—call 
them “structural reasons”—to have her mind in good 
coherent order. You have a reason not to have contra- 
dictory beliefs, a reason to intend means to ends that 
you intend, and so on. The view is that rationality con- 
sists in responding correctly to your structural reasons. 
Responding correctly to structural reasons would super- 
vene on your mind, so this view is immune to the quick 
objection. 

But it misunderstands responding correctly to rea- 
sons. To respond correctly to reasons, you must F when 
your reasons together require you to F, not when you 
have a single reason to F. Even if you have a structural 
reason not to have contradictory beliefs, you might have 
another reason to have contradictory beliefs. For exam- 
ple, an evil demon might announce it will destroy the 
world unless you have some contradictory beliefs. In a 
case like this, your reasons together may require you to 
have contradictory beliefs, so that responding correctly 
to reasons would imply having contradictory beliefs. 
Nevertheless, if you do have contradictory beliefs, you 
will not be fully rational. This shows that rationality 
is not a part of responding correctly to reasons, even if 
structural reasons indeed exist. 

This argument illustrates a fundamental difficulty 
that stands in the way of reducing rationality to reasons. 
Rationality imposes strict requirements on us, and if we 
violate them, we are necessarily not fully rational. But 
what reasons require of us is generally defeasible; it can 
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be overridden by further reasons. So reasons are not well 
suited to account for rationality. 


4.2 Myth Theory 

Another version of the view that rationality is entailed 
by responding correctly to reasons is known as “myth 
theory.”"” It is the view that rationality in the mental 
sense is a myth, or at least that structural rationality is a 
myth. Structural rationality is the property a person has 
when she has consistent beliefs and intentions, intends 
means to ends she intends, and so on. Myth theorists do 
not deny that this property exists. But they think it is 
an uninteresting property, because if a person responds 
correctly to reasons, she will possess it automatically as 
a consequence. They think that, if your mind is properly 
aligned with the world—so you believe what your rea- 
sons require you to believe, you intend whatever your 
reasons require you to intend, and so on—a necessary 
consequence is that your mind will be properly aligned 
internally. You will have consistent beliefs and inten- 
tions, intend means to ends you intend, and so on; you 
will be structurally rational. 

Kolodny (2007) expresses his version of myth theory 
by denying that “there are rational requirements of 
formal coherence as such.” Rationality definitely has 
requirements in one sense. Any necessary condition for 
something to possess a property may be called a require- 
ment of the property. For example, a necessary condi- 
tion for being bald is not having much hair, so we may 
say that baldness requires you not to have much hair. In 
this sense, rationality definitely requires you not to have 
contradictory beliefs. But Kolodny is using “requires” 
in a different sense. This is the sense that appears in 
my expression “your reasons require you to F.” To say 
rationality requires you to F is to say that rationality 
prescribes Fing to you. Kolodny denies that rationality 
issues prescriptions. 

I have two replies to myth theory. One is to deny it. 
I deny that if your mind is properly aligned with the 
world, it will necessarily be properly aligned internally. 
An example is where your reasons permit you to do 
something and also permit you not to do it. Cases like 
this are common: your reasons for going to Paris may 
neither outweigh nor be outweighed by your reasons for 
not going to Paris. Then, even if your mind is properly 
aligned with the world, you may intend to go to Paris, 
and also you may intend not to go to Paris. Furthermore, 
the world may give you no reason for not having both 
intentions; having both might even be helpful because 
it leads you to prepare for both eventualities. So, even if 
your mind is properly aligned with the world, you may 
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have both intentions. But then your mind is not prop- 
erly aligned internally: you are not fully rational if you 
have contradictory intentions. You respond correctly 
to reasons, but you are not fully rational. In reaction to 
examples like this, Kolodny (2007) urges us to abandon 
the idea that you are necessarily not fully rational if you 
have contradictory intentions. That seems to me a des- 
perate expedient. 

The second reply is to point out that often you can- 
not respond correctly to reasons except by engaging 
your rationality. For instance, if you are to intend means 
to an end you intend, you may need to work out by 
theoretical reasoning what is a means to your end, and 
you may then need to do some instrumental reasoning 
in order to come to intend the means. Reasoning is a 
rule-governed process that takes you from some exist- 
ing premise-attitudes of yours, such as existing beliefs 
and intentions, to a new conclusion-attitude. Correct 
reasoning is reasoning that follows correct rules. What 
rules are correct is determined by principles of rational- 
ity that connect the conclusion-attitude to the premise- 
attitudes.’* These principles are independent of what 
your reasons require of you. They have to be indepen- 
dent, because reasoning proceeds in exactly the same 
way whether or not your reasons require you to have the 
premise-attitudes or the conclusion-attitude. You can 
reason equally well from false beliefs and bad intentions 
as from true beliefs and good intentions. 

So even if it were true that responding correctly to 
reasons entails rationality, it would not follow that 
rationality can be reduced to responding correctly to 
reasons. Responding correctly to reasons itself depends 
on rationality. 


5. Rationality as Responding Correctly to Beliefs 
about Reasons 


A different reductive claim is that rationality consists 
in responding correctly to beliefs about reasons. This is 
subject to various interpretations. According to one pro- 
posed by Parfit (2011, chapter 5), responding correctly 
to beliefs about reasons implies Fing whenever you are 
required to F by the reasons you believe there to be. 
But this is ruled out by the first objection in section 3, 
because what is required by the reasons you believe there 
to be does not supervene on your mind. The way these 
reasons combine together to determine what is required 
may depend on something external to you. 

On a second interpretation, supported by Kolodny 
(2008b), responding correctly to beliefs about reasons 
implies Fing whenever you believe your reasons require 
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you to F. Because your belief is a mental state, it super- 
venes on your mind, so this interpretation is immune 
to the first objection. We can make it immune to the 
second objection by confining the response to mental 
states: we may say that rationality consists in having 
mental states that are correct responses to beliefs about 
reasons. 

A third interpretation takes responding correctly to 
beliefs about reasons to imply intending to F—rather 
than actually Fing—when you believe your reasons 
require you to F. Once again, this is immune to the first 
objection, because your belief is a mental state. It is also 
immune to the second objection, because intending to F 
is a mental state and so supervenes on your mind. 

Neither the second nor the third interpretation is 
vulnerable to the quick objection. Nevertheless, both 
are mistaken. The problem with them is that they do 
not cover all of rationality. There are many necessary 
conditions for rationality that are not implied by this 
claim. For example, you are necessarily not fully rational 
if you have contradictory beliefs or intentions, even if 
you yourself believe there is nothing wrong with having 
contradictory beliefs or intentions. These conditions of 
rationality impose “strict liability,” as I put it (Broome, 
2013, p. 75). 

Still, the third interpretation of the claim does con- 
tain a truth. Rationality does not consist in responding 
correctly to beliefs about reasons, but it does require 
responding correctly to beliefs about reasons. That is: 


Rationality requires of you that you intend to F if you 
believe your reasons require you to F. 


This is one among many requirements of rationality. 
I call it enkrasia. It is only a rough formulation of enkra- 
sia; an accurate formulation is more complicated and 
appears in Broome (2013, pp. 170-171). The state of 
believing your reasons require you to F while not intend- 
ing to F is known as akrasia. Akrasia has traditionally 
been taken to be irrational (Aristotle, Nicomachean Eth- 
ics, Book 7; Davidson, 1969), and enkrasia asserts that it 
is irrational. 


6. Conclusion 


Enkrasia is an important connection between reasons 
and rationality. It is a kernel of truth that is hidden 
inside the grander reductive views I have argued against: 
the view that rationality consists in responding correctly 
to reasons and the view that it consists in responding 
correctly to beliefs about reasons. Those reductive views 
are false. 
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Notes 


1. Leading works in this movement are Nagel (1970), Parfit 
(2011), and Scanlon (1998). 


2. See the definitions in Broome (2013, sections 4.2 and 4.3). 


3. For example, by Scanlon (2007) and Wallace (2003/2018). 
In chapter 10.5 by Nida-Rtiimelin, Gutwald, and Zuber (this 
handbook), the term is used differently. 


4. I believe this term originates with Max Weber; see Kalberg 
(1980). 


5. I assume you have no attitude that favors turning left. 
Arpaly (2003) argues that sometimes it is genuinely rational 
to do something you believe you ought not to do. Her prime 
example is Huck Finn, who believes he ought not to conceal 
the escaped slave Jim but does so. She claims his decision is 
rational because it coheres well with other attitudes of Huck’s 
apart from his belief. My case is not like that. 


6. The most thoroughgoing defence of this view is Kiesewet- 
ter’s (2017, chapter 7). Other examples are in Gibbard (1990, 
p. 161) and Lord (2017). Lord’s view is that rationality consists 
in doing what you ought to do, but it will quickly appear that 
this amounts to the same thing. 


7. See Broome (2013, chapter 5), where parts of the following 
argument are developed in more detail. There is also a fuller 
development in Broome (2021). 


8. This is Kiesewetter’s (2017) and Lord’s (2017) interpretation. 
9. There is a full discussion of the argument in Broome (2021). 


10. Kurt Sylvan pointed out to me that it is defended by Prichard 
(2002, pp. 95-97). Even with Jonathan Dancy’s help, I have not 
been able to extract a credible argument from Prichard’s text. 


11. The following arguments are set out more fully in Broome 
(2013, section 5.4). 


12. The leading proponents are Kolodny (2008a) and Raz (2005S). 
13. Specifically by what I call “basing permissions of rational- 
ity” (see Broome, 2013, sections 13.7 and 14.2). 
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2.2 Practical and Theoretical Rationality 
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Summary 


Philosophers have long distinguished between practical 
and theoretical rationality. The first section of this chap- 
ter begins by discussing the ways in which this distinc- 
tion was drawn by Aristotle and Kant; then it sketches 
what seems to be the general consensus today about 
how, at least roughly, the distinction should be drawn. 
The rest of this chapter explores what practical and theo- 
retical rationality have in common: in the second sec- 
tion, several parallels between practical and theoretical 
rationality are outlined, and it is argued that these paral- 
lels make it plausible that a unifying account of rational- 
ity can be given. Finally, in the third section, a number 
of such unifying accounts of practical and theoretical 
rationality are surveyed. These include accounts that are 
inspired in various ways by Hume and by the results of 
formal decision theory, as well as views that appeal to 
reasons and to the distinctive value of correct or appro- 
priate attitudes. 


1. The Distinction between Practical 
and Theoretical Rationality 


1.1. Aristotle and Kant 
The terms “practical” and “theoretical” derived from 
ancient Greek philosophy, particularly from the work of 
Aristotle, who in his ethical writings gave a theory of 
both theoria and praxis. 

In the relevant senses,’ both thedria and praxis are 
“activities” (Nicomachean Ethics, 1146b33) of “the parts 
of the soul that involve reason” (1095a10). In Aristotle’s 
teleological system, the nature of both these activities 
is explained in terms of their essential function—that 
is, the role that these activities ideally should play in a 
human life. Specifically, when it fulfills its essential 
function, theoria—often translated as “contemplation” 
or “study”—is the manifestation of the virtue of theoreti- 
cal wisdom (sophia, 1177a26), which in turn is explained 
as consisting of scientific knowledge (epistémé), resting 


on comprehension (nous) of the first principles of science 
(1141a18). Theoria, then, involves actively exercising 
one’s understanding of a body of scientific knowledge 
that one possesses (1177426). 

However, theoretical wisdom (sophia) is only one of the 
so-called intellectual virtues. The intellectual virtues also 
include technical skill (techné, 1140a1-25) and practical 
wisdom or prudence (phronésis, 1140a26-b29). While the- 
oretical wisdom is manifested in contemplation (thedria), 
practical wisdom is manifested in action (praxis). At least 
when the agent’s capacities are all fulfilling their essential 
functions, a praxis is the execution of a “decision” (pro- 
hairesis, 1094a5). According to Aristotle, a “decision” is a 
special kind of desire to perform an action here and now, 
which arises from a wish (boulésis, 1111b26) for an end 
that is viewed as good, through deliberation or rational 
calculation (logismos) about how best to achieve that end 
(1112b15). When we manifest the virtue of practical wis- 
dom, we make a correct decision, on the basis of grasp- 
ing the truth about the good and correctly perceiving our 
practical situation, and by executing such a decision, we 
perform a correct action (1140b21). 

This is the general picture of Aristotle’s distinction 
between theoretical and practical wisdom, as well as 
the activities, theoria and praxis, that under favorable 
circumstances are their manifestations. However, at one 
point, he seems to give a rather different general char- 
acterization of the difference between theoretical and 
practical reason (1139a3-15): 


Let us assume there are two parts [of the soul] that have 
reason: with one we study beings whose principles do not 
admit of being otherwise than they are, and with the other 
we study beings whose principles admit of being other- 
wise. .. . Let us call one of these the scientific part, and the 
other the rationally calculating part; for deliberating is the 
same as rationally calculating, and no one deliberates about 
what cannot be otherwise. Hence the rationally calculating 
part is one part of the part of the soul that has reason.” 


Here Aristotle seems to say that theoretical reason is con- 
cerned with necessary truths, and practical reason—the 
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sort of reason that consists in the capacity for “delib- 
erating”—is concerned with contingent truths. This 
characterization strikes contemporary philosophers as 
puzzling—since it leaves no room for theoretical reason- 
ing about contingent facts that have no practical rele- 
vance (such as the contingent facts of ancient history, 
paleontology, and the like). 

In a similar way, Kant also seems to seek an account 
of the difference between practical and theoretical rea- 
son in terms of the “cognitions” (Erkenntnisse) that they 
are concerned with. One short statement of this distinc- 
tion appears in the presentation of Kant’s Logic that was 
published toward the end of Kant’s life by his colleague 
Gottlob Jasche. In the primary sense, according to Kant 
(1900ff., volume 9, p. 86), a “practical cognition” is an 
“imperative”—where an imperative is defined as “a prop- 
osition that expresses a possible free action, whereby a 
certain end is to be made real.” By contrast, theoretical 
cognitions are characterized as follows: 


Theoretical cognitions are ones that express not what ought 
to be but rather what is, hence they have as their object not 
an acting (kein Handeln) but rather a being (ein Sein).° 


Kant (1900ff., volume 4, p. 413) gives a different 
account of “imperatives” in his Groundwork of the Meta- 
physics of Morals, where he says that an imperative is the 
“formula” of “a representation of an objective principle, 
so far as it is necessitating for a will.” What this seems 
to mean is that an imperative is the expression of a cog- 
nition that represents a principle that the will of every 
rational being ought to follow—or, more precisely, the 
application of such a principle to an agent who does not 
necessarily follow or conform to this principle. As he says, 
“All imperatives are expressed through an ought... . 
They say that it would be good to do or refrain from 
something” (p. 413). 

This approach, however, brought Kant into difficul- 
ties with his account of so-called hypothetical impera- 
tives (p. 414). Every hypothetical imperative presupposes 
a certain end or purpose and merely indicates the means 
that are necessary for achieving that end or purpose. In 
other words, the imperative asserts that a certain action is 
good for the relevant end. Like all imperatives, each hypo- 
thetical imperative represents a principle that should in 
a sense guide the will of all rational beings—at least in 
the weak sense that no rational beings may will the end 
without also willing the necessary means. 

However, as Kant later pointed out in the “First Intro- 
duction” to the Critique of the Power of Judgment, each 
of these hypothetical imperatives is in a way nothing 
more than a theoretical proposition about the causal 
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connection between an action and the relevant end. His 
puzzling conclusion is that hypothetical imperatives— 
which he now prefers to call “technical” rather than 
“hypothetical”—are practical “in form” but not “in con- 
tent” (Kant, 1900ff., volume 20, pp. 196, 200). 

These difficulties seem to reveal that it is a mistake 
to account for the difference between practical and 
theoretical reason in terms of two kinds of proposi- 
tions (or “cognitions”). Instead, the difference should be 
explained by distinguishing between two kinds of atti- 
tudes that can be taken toward propositions. This is the 
approach that has become the consensus among con- 
temporary philosophers. 


1.2 Belief versus Practical Attitudes 

Virtually all contemporary philosophers seem to agree 
on the following way of distinguishing between theo- 
retical and practical rationality: 


(a) Theoretical rationality is the rationality of beliefs 
and other belief-like or belief-involving phenomena 
(such as credences, inferences, and the like). 


(b) Practical rationality is the rationality of the practical 
attitudes—like intentions, decisions (or choices), and 
preferences—that are directly executed in action. 


According to this approach, theoretical and practical 
rationality do not differ with respect to the propositions 
that they are concerned with. They differ with respect 
to the kinds of attitude that are in question—(a) belief 
in the case of theoretical rationality and (b) intention or 
decision or preference in the case of practical rationality. 

In appealing to this contrast between (a) belief and 
(b) intention or decision, are contemporary philoso- 
phers departing from the philosophical tradition? There 
are reasons for thinking that they are not. As we have 
seen, Aristotle firmly associates theoretical wisdom with 
knowledge, and many contemporary philosophers think 
of knowledge as in a sense the best case of a certain cen- 
tral kind of belief—specifically, of the kind of belief that 
is sometimes called “full belief.” On the practical side, 
Aristotle clearly associates practical wisdom with deci- 
sion and action. Similarly, Kant (1900ff., volume 4, 
p. 412) states that “the will is nothing other than prac- 
tical reason”—and what Kant calls a “determination of 
the will” seems essentially to involve an intention to act. 

Still, on both the theoretical and the practical side, 
several different kinds of attitudes could be considered. 
On the theoretical side, there is not just full or outright 
belief but also various levels of confidence or credence 
that we can take toward the propositions that concern 
us. In addition, there may be various further belief-like 
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attitudes that should be considered. Some philosophers— 
like Joyce (2010)—have suggested that there are both 
precise and imprecise credences; some—like Edgington 
(1995)—have argued that there are both conditional and 
unconditional credences; and others—like Friedman 
(2013)—have argued that suspension of judgment is a 
distinctive belief-like attitude as well. If all these differ- 
ent belief-like attitudes exist, they can all be assessed as 
rational or irrational. If any attitudes of these kinds are 
rational, they would count as theoretically rational. 

In addition to these belief-like mental states, there are 
also belief-involving events of belief revision or theoreti- 
cal reasoning. Such events involve changing one’s beliefs 
in some way—either by coming to have a belief-like 
attitude toward a proposition toward which one previ- 
ously had no attitude at all, by replacing an old belief- 
like attitude that one had in a proposition with a new 
attitude toward the very same proposition, or simply by 
reaffirming the old attitude that one had toward that 
proposition. If any of these mental events are rational, 
they would also count as theoretically rational. 

On the practical side, intentions are mental states— 
while choices or decisions are mental events in which 
one forms or revises one’s intentions about how to act. 
If an intention is executed or carried out, the execution 
of the intention is an action. It seems that all of these 
states and events—intentions, decisions (or choices), 
and actions—can be assessed as rational or irrational. If 
rational, they would count as practically rational. Some 
philosophers have argued that, among these states and 
events, some are more fundamental to practical ratio- 
nality than others. For example, Harman (1986, p. 77) 
has argued that practical rationality fundamentally con- 
cerns processes in which agents changes their plans or 
intentions, and Broome (2013, p. 250) has argued that 
requirements of rationality fundamentally apply to sets 
of mental states, like beliefs and intentions, rather than 
applying directly to actions. 

In formal accounts of practical rationality—such 
as those of Jeffrey (1983) and Joyce (1999)—the focus 
is typically not on intentions, decisions, or actions 
but on preferences. In these accounts, the fundamental 
instances of practical rationality are preferences and 
events in which our preferences change in response to 
changes in our beliefs. This approach is also akin to the 
contemporary consensus, since theorists who focus on 
preferences typically assume that there is an intimate 
connection between preferences, on the one hand, and 
choices (or decisions) and actions, on the other. Spe- 
cifically, these theorists typically assume that no agent 
ever chooses (or decides on), or intentionally takes, any 
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course of action A if there is an available alternative B 
that the agent prefers over A. 

Admittedly, some contemporary philosophers would 
have reason to reject this way of distinguishing prac- 
tical and theoretical rationality. Specifically, some 
philosophers—such as Setiya (2007, p. 49)—actually iden- 
tify intentions with a special kind of belief. On this view, 
for you to intend to F is just for you to have a belief of a 
certain kind that you will F. What is special about inten- 
tion, on this view, is that it is a belief that tends in a certain 
distinctive way to cause the truth of its propositional con- 
tent (unlike most beliefs, which are caused by the truth of 
their content or else causally independent of the truth of 
their content). On this view, it could happen that when 
you have a belief of this sort, to the effect that you will 
F, this belief is theoretically rational (if when you have the 
belief, it is clear to you, given your evidence, that you will 
indeed F), even if it is not practically rational (if Fing is 
obviously a foolish or inadvisable course of action). For 
this reason, these theorists would have to find a differ- 
ent way to distinguish between practical and theoretical 
rationality. 

Otherwise, however, virtually all contemporary theo- 
rists would distinguish between practical and theoretical 
rationality in the way that I have described. According to 
this contemporary consensus, then, theoretical rational- 
ity is the rationality of beliefs, while practical rationality 
is the rationality of intentions and choices and the like. 


2. Unifying Practical and Theoretical Rationality? 


Practical and theoretical rationality have some striking fea- 
tures in common. First, similar vocabulary is used on both 
the practical and the theoretical sides. The term “rational” 
itself is applicable, in what appears to be the very same 
sense, to both beliefs and choices. We can talk about the 
beliefs and choices that are “rationally required” of a 
thinker in a given situation and of the beliefs and choices 
that are “rationally permissible” for the thinker in the situ- 
ation. We can talk about “reasonable” or “justified” beliefs 
and choices—where there is one way of using terms like 
“reasonable” and “justified” on which they express the 
very same concept as the term “rational.” Finally, we can 
talk about “reasons” for beliefs and “reasons” for choices 
and actions; on both the practical and theoretical sides, we 
distinguish between there being “some reason” for a belief 
or an action, “sufficient reason” for the belief or action, 
and “compelling” or “overriding reason” for the belief 
or action. It seems unlikely that these terms—“justified,” 
“reasonable,” and “reason”—are used in different senses 
when applied to belief as opposed to choice or action. 
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Moreover, both theoretical and practical rationality 
come in degrees: while some beliefs are only slightly irra- 
tional, other beliefs are extremely irrational—and the 
same is obviously true of choices or decisions as well. 
On both the theoretical and the practical side, there is 
some plausibility in the following definition of a “ratio- 
nal requirement”: meeting a certain condition C is ratio- 
nally required of you at a time ¢ if and only if, in all 
relevantly available worlds in which you are thinking as 
rationally as you can at t, you meet condition C.* 

Finally, on both the practical and the theoretical side, 
we can draw the same distinction between your having 
beliefs (or intentions) that it is rational for you to have 
and your having those beliefs (or intentions) rationally. 
An irrational agent might, through sheer dumb luck, 
choose an option that it is rational for him to choose, 
but he would not thereby be choosing this option 
rationally. In epistemology, this distinction is typically 
drawn by means of the terminology of “propositional” 
and “doxastic” justification: an irrational thinker might, 
through sheer dumb luck, believe a proposition that 
there is propositional justification for her to believe, but 
this irrational thinker would not thereby be believ- 
ing the proposition in a doxastically justified mannet. 
Clearly, however, the same distinction can be drawn 
among choices and intentions as well. 

With all these parallels between practical and theo- 
retical rationality, it seems plausible that there should 
in principle be some way of unifying the two domains. 
What would such a unification of these two domains 
amount to? First, it would consist of a collection of 
general principles, capturing features of rationality in 
general, including both practical and theoretical ratio- 
nality. Second, it would have to be true that the specific 
features of theoretical rationality can all be derived by 
combining these general principles about rationality 
with some specific principles about belief, while the spe- 
cific features of practical rationality can all be derived 
by combining these general principles about rationality 
with some specific principles about intention, choice, 
and action. As we shall see in the following section, a 
number of different attempts have been made to provide 
a unifying account of this sort. 


3. Unifying Accounts of Rationality 


3.1 Formal Theories of Rationality 

Philosophers who have devised formal theories of ratio- 
nality have typically offered unified accounts of both 
practical and theoretical rationality. One particularly 
common idea is that to be rational, an agent’s beliefs 
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and preferences must be capable of being “represented” 
by a probability function and a utility function. There 
are some subtle differences between different versions of 
this idea, but according to the most common version, 
to say that a probability function and utility function 
“represent” the agent’s beliefs and preferences is to say 
that the following conditions hold: 


(a) For any two relevant prospects A and B, the agent 
prefers A over B if and only if the utility function 
assigns a higher utility to A than to B, and the agent 
is indifferent between A and B if and only if the util- 
ity function assigns the same utility to A as to B. 

(b) For any two propositions p and q, the agent believes 
p with greater confidence than q if and only if the 
probability function assigns a higher probability to 
p than to q, and the agent believes p and q with the 
same degree of confidence if and only if the probabil- 
ity function assigns the same probability to p as to q. 

(c) For every prospect A, the utility that the utility 
function assigns to A is equal to A’s expected utility 
according to this probability function. 


Suppose that every rational agent’s beliefs and prefer- 
ences can be represented by a utility function and prob- 
ability function in this way. Then it follows that every 
rational agent’s preferences give a complete ordering of 
the prospects over which they are defined: that is, for 
any two such prospects A and B, the agent prefers A over 
B, prefers B over A, or is indifferent between the two. 
Rational agents’ preferences must also be transitive: if the 
agent prefers A over B and either prefers B over C or is 
indifferent between B and C, then the agent must also 
prefer A over C. 

The centerpiece of many such formal accounts of 
rationality is a “representation theorem”—a formal 
proof that so long as the agent’s beliefs and preferences 
meet certain conditions, they can be “represented” by 
a probability function and utility function in some- 
thing like this sense. One of the most powerful of these 
theorems is due to Savage (1972). Savage only needs to 
assume that the agent’s preferences meet a number of 
conditions—no special assumptions about the agent’s 
beliefs are required—and on this basis he proves that 
these preferences can be represented by a unique prob- 
ability function and a utility function that is unique up 
to an arbitrary choice of a unit and zero point. 

How could this account of rationality unify practical 
and theoretical rationality? One approach might rely on 
Savage’s theorem and attempt simply to reduce theoretical 
rationality to practical rationality. If it is true that every 
rational agent’s preferences must satisfy the conditions 
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that Savage’s proof assumes, then a rational agent’s beliefs 
can simply be identified with the probability function that 
is uniquely determined by these preferences. This might 
suggest a pragmatist view of belief—in effect, the view that 
a practically rational agent has no beliefs except for those 
that are already implicit in her preferences. If this pragma- 
tist view of belief is correct, and if there is no more to be 
said about rational belief than that rational beliefs must 
be capable of being represented by a probability function, 
then this approach would be able to reduce theoretical 
rationality to practical rationality in this way. 

However, this pragmatist view of belief—the view 
that the practically rational agent’s beliefs are just those 
that are implicit in her preferences—is intensely contro- 
versial. Most more recent proponents of various forms 
of formal decision theory—such as Joyce (1999, p. 90)— 
would prefer to avoid being committed to this pragma- 
tist view of belief. 

If we reject the pragmatist view of belief, we would 
presumably accept that there are principles of rational 
belief that do not simply flow from the constraints on 
rational preferences. So, we would need an alternative 
way of unifying theoretical and practical rationality. 
One approach might stay close to the pragmatist tradi- 
tion by arguing that the requirements of theoretical and 
practical rationality are both explained by the pragmatic 
defects of violating those requirements. For example, 
we might rely on the well-known “money pump” and 
“Dutch book” arguments, which seek to show that vio- 
lating any requirements of either practical or theoreti- 
cal rationality makes us willing to accept a collection of 
bets that will result in our losing money no matter what 
happens.° 

These “Dutch book” and “money pump” arguments 
are fascinating, but their significance is disputed. Even if 
your beliefs do make you willing to accept a collection 
of bets that would result in your losing money whatever 
happens, you might never be offered such a collection 
of bets, and your beliefs might provide extremely useful 
guidance for the practical situations that you are actu- 
ally in. So, it is controversial whether these arguments 
can really show what fundamentally unifies practical 
and theoretical rationality. 


3.2 Reducing Theoretical Rationality 

to Instrumental Rationality? 

Many contemporary philosophers who work on ratio- 
nality seek to develop certain key insights of Hume 
(1739/1740, ILiii.3). The interpretation of Hume is con- 
troversial. However, some contemporary philosophers 
take Hume’s key lesson for us to be that the primary 


141 


form of rationality is what is often called “instrumen- 
tal rationality”—the rationality of taking what (at least 
according to the evidence) seem to be optimal means to 
one’s goals. 

If instrumental rationality is primary in this way, 
then it appears that theoretical rationality must consist 
in having a set of beliefs that (according to the evidence) 
seems to be an optimal means to our distinctive cognitive 
or epistemic goals. Most versions of this approach assume 
that the relevant cognitive goal is that of believing the 
truth, and not believing what is false, about the questions 
that we have considered. It is then rationally permissible 
for a thinker to have a given set of beliefs if and only if 
these beliefs seem (at least according to the evidence) to 
be optimal means to this goal.” 

This approach has also been criticized by a number of 
philosophers—notably, by Thomas Kelly (2003). Kelly’s 
main concern is that it does not seem true that ordinary 
thinkers have very general cognitive aims or goals. It is 
not obviously true that it genuinely is one of my goals 
simply to believe true propositions as such. My cognitive 
goals tend to be more local and specific. I have the goal 
of having a true belief about the times of the flights from 
Denmark to California—but this is only because I need 
to make a choice about how to fly from Denmark to 
California sometime soon. I have no comparable inter- 
est in having true beliefs about the times of the flights 
between Cairo and Moscow, since I currently have no 
reason to think that I am likely to fly on that route at 
any point in the foreseeable future. 

If Kelly (2003) is right that our cognitive goals are 
local and specific in this way, then these goals are idio- 
syncratic. Different people will have different cognitive 
goals. But the requirements of theoretical rationality 
seem to be in a way intersubjective. If you are compelled 
by an annoying interlocutor to consider a certain propo- 
sition p, and at the same time you are confronted with 
compelling evidence for p, then it seems that theoretical 
rationality will require you to believe p—even if it was 
not in any sense one of your goals to have a true belief 
about p. According to Kelly (2003, p. 623), it is hard to 
see how the instrumentalist conception of theoretical 
rationality can explain this fact. 

For these reasons, then, while the project of reduc- 
ing theoretical rationality to practical rationality is still 
worth investigating, few contemporary philosophers are 
optimistic about its chances of success. 


3.3. Rationality and Reasons 
Many contemporary philosophers seek to understand 
rationality in terms of reasons. Roughly, to think rationally, 
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on this reasons-centered approach, is to respond appro- 
priately to one’s reasons. This approach holds out the 
hope of unifying practical and theoretical rationality. 
Practical rationality consists in making choices—and 
forming, revising, and maintaining one’s intentions or 
plans about how to act—in a way that involves respond- 
ing appropriately to reasons for such choices and 
intentions. Theoretical rationality consists in forming, 
revising, and maintaining one’s beliefs in a way that 
involves responding appropriately to reasons for belief. 

A number of variations on this basic theme are pos- 
sible. Thus, for example, Joseph Raz does not directly 
identify rationality with responding appropriately to 
reasons. Instead, Raz (2011, p. 89) identifies irrationality 
with the “malfunctioning” of our “rational powers”— 
and Raz (2011, p. 85) identifies these “rational pow- 
ers” with our capacities for recognizing and responding 
appropriately to reasons. Kolodny (2005, 2008) hopes to 
reduce rationality to responding correctly to the reasons 
that one believes there to be. 

A different refinement of this approach postulates 
a distinction between “objective” and “subjective” (or 
“apparent”) reasons. On one interpretation of this dis- 
tinction, there is a certain set of propositions—call them 
the “reason-propositions” for the relevant agent at the 
time in question—such that the “objective reasons” for 
the agent at that time can be identified with the reason- 
propositions that are true, and the “subjective reasons” 
that the agent has at that time can be identified with the 
reason-propositions that the agent believes at that time. 
Then we could say that rationality consists of responding 
appropriately to one’s subjective (or apparent) reasons.® 

The attempt to give a unified account of rationality in 
terms of responding appropriately to reasons has been crit- 
icized in many ways. (For some powerful and interesting 
criticisms, see chapter 2.1 by John Broome, this handbook, 
and also Broome, 2013, chapters 5 and 6.) Perhaps the 
most fundamental objection, however, is the following: 
How exactly does this account of rationality as responding 
appropriately to (“subjective”) reasons succeed in unify- 
ing practical and theoretical rationality? Is the notion of a 
“reason” itself a sufficiently unified notion? 

According to Raz (2011, p. 36), epistemic reasons for 
believing a proposition are so “through being facts that 
are part of a case for (belief in) its truth.” Even if there 
are also nonepistemic reasons for belief (for example, if 
an evil demon credibly threatens to punish me unless I 
believe something), the epistemic reasons are, in his ter- 
minology, “standard reasons” (Raz, 2011, p. 40). By con- 
trast, Raz (1999) says that reasons for actions “are facts in 
virtue of which those actions are good in some respect 
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and to some degree” (p. 23). In other words, standard 
reasons for belief have to do with truth, while reasons for 
action have to do with value, and Raz (2011, pp. 42-43) 
denies that truth, or true belief, is a value. 

For Raz (1999, pp. 22-23), standard reasons are uni- 
fied by the role that they play in guiding our thought 
and by the principle that it is constitutive of being an 
intelligible thinker and agent in the first place that one 
has a general disposition to act in ways that one believes 
to be good, as well as to believe propositions that are 
supported by truth-related considerations. Raz’s ideas 
on this topic are controversial: some philosophers, like 
Setiya (2007, pp. 21-67), argue strenuously against Raz’s 
idea that intentional action is always undertaken under 
“the guise of the good.” So, it is debated whether Raz 
successfully answers the unification challenge. 

Schroeder (2007, p. 29) takes a different approach. 
He aims to combine the appeal to reasons with the idea 
that is suggested by Hume’s (1739/1740, ILiii.3) focus on 
desire. Thus, he proposes that all reasons are grounded 
in desires (or in some other similar attitude). Every rea- 
son for you to F is grounded in some state of affairs S$ 
that you desire, in such a way that the reason consists of 
some fact that helps to explain why your Fing promotes, 
or raises the chances of, S (compared to the relevant 
alternative to Fing). Schroeder (2007, p. 113) expresses 
sympathy for the idea that a view of this kind gives a 
good account of reasons for belief as well as of reasons 
for action. 

On the face of it, this is a surprising account of reasons 
for belief. But Schroeder ingeniously argues for several 
auxiliary theses that could explain why this view is more 
plausible than it first appears. First, a reason for believ- 
ing p only needs to explain why believing p raises the 
chances of some desired state of affairs to some degree 
(compared to the relevant alternative). So, as Schroeder 
(2007, p. 113) argues, whatever states of affairs the agent 
may desire, if a proposition p is false, then not believing 
p will at least somewhat raise the chances of these desired 
states of affairs; so there is always some reason for not 
believing p if p is false. Second, he argues that the weight 
of a reason is not automatically determined either by 
the strength of the desire or by the degree to which the 
reason raises the chances of the desired state of affairs— 
instead, he proposes an entirely different account of the 
weight of reasons (Schroeder, 2007, chapter 7); so even if 
all reasons for belief are grounded in desires, it may still 
normally be the case that there is most reason for us not 
to believe p if p is false. 

Schroeder’s Hume-inspired account of reasons is still 
a work in progress. It remains to be seen exactly how an 
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account developed along these lines can accommodate 
enough of our intuitive judgments about the rationality 
of belief, choice, and action. Still, if an account of this 
kind can be developed in detail, it would be an illuminat- 
ing way of unifying practical and theoretical rationality. 


3.4 A Values-Centered Approach 

As we saw in section 2, rationality comes in degrees. Of 
the alternative possible beliefs and choices that you might 
have in response to your cognitive situation, some are 
more irrational than others. In this sense, rationality 
“ranks” some beliefs and choices as more rational than 
others. 

This ranking of beliefs and choices will typically play 
a reasoning-guiding role: normally, your judgments of 
rationality will guide you toward having the beliefs or 
making the choices that you judge to be more rational 
and away from the beliefs and choices that you judge to 
be less rational. In this respect, rationality resembles a 
value, or a way of being good. (Specifically, it resembles 
the values that rank some alternatives as in the relevant 
way better than others and guide us toward preferring the 
alternatives that we judge to be better over those that we 
judge to be worse.) Thus, it seems plausible that rational- 
ity is itself a value. Rational thinking is thinking that is 
good in a certain way; irrational thinking is thinking that 
is bad in a corresponding way, and in general, the more 
irrational your thinking is, the worse your thinking is in 
this way.” 

While we can evaluate both beliefs and choices as 
rational, there are also other ways of evaluating beliefs 
and choices. There is one particularly fundamental way 
of evaluating beliefs and choices that I shall pick out by 
using the terms “correct” and “incorrect,” or “appropri- 
ate” and “inappropriate.” (In everyday language, many 
other terms are also used: for example, we could talk 
about your “getting things right” in your belief about 
a certain question, or about your making “the wrong 
choice” among a set of options, and so on.) 

As I am using the terms here, the key difference 
between rationality and appropriateness (or correct- 
ness) is this: whether or not a belief or choice is rational 
depends solely on its relations to the mental states and 
events that are present in the thinker’s mind at the rel- 
evant times—whereas whether or not a belief or choice 
is appropriate (or correct) depends on its relations to 
the external world. Thus, whether a belief is correct or 
appropriate depends on whether the proposition that is 
believed really is true, and whether a choice is correct or 
appropriate depends on whether the chosen option is 
feasible and really a good thing to do. 
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Appropriateness also seems to come in degrees: some 
choices are more inappropriate—more badly wrong or 
incorrect—than others. I propose that something simi- 
lar is true of beliefs. Specifically, I propose that what 
formal epistemologists like Joyce (1998) refer to as a 
belief’s “degree of inaccuracy” can be interpreted as a 
measure of the degree to which a belief is incorrect or 
inappropriate. Thus, if the proposition p is true, then 
the greater the confidence with which you believe p, 
the more appropriate or correct your belief is, while if 
p is false, then the greater the confidence with which 
you believe it, the more inappropriate or incorrect your 
belief is. (So, on this interpretation of what it is to “get 
things right” or to “believe appropriately,” it is possible 
for an irrational belief to get things right, through sheer 
dumb luck.) 

In reasoning, we can form expectations of how cor- 
rect different alternative beliefs or choices would be, and 
these expectations can guide us toward the beliefs and 
choices that we expect to be more correct and away from 
those that we expect to be less correct. In this way, it 
seems that the concept of correct or appropriate attitudes 
is—like the concept of rational attitudes—an alternative- 
ranking reasoning-guiding concept. This gives us reason 
to think that it is also a value concept. 

It may be plausible that correctness is, to put it meta- 
phorically, the “aim” of rationality. Suppose that for 
every agent and every point in time, there is a special 
probability function P such that the credences the agent 
has at that time are rational precisely in proportion to 
how closely they approximate that probability function 
P. Let us call this the “rational probability function” for 
the agent at that time. Then the key connection between 
rationality and correctness could be this: irrationality is, 
according to this rational probability function, bad news 
about correctness. The more irrational your beliefs or 
choices are, the greater the degree to which this prob- 
ability function expects your beliefs and choices to fall 
short of the highest expected degree of correctness that 
is available to you in your situation. 

Some philosophers will certainly object to this 
approach. As we have seen, Raz (2011, p. 40) denies that 
true belief is a value, and it may seem strange to say that 
correctness or appropriateness is a value. It must be con- 
ceded that it is a distinctive kind of value: the value of 
being admirable is exemplified by the objects of admira- 
tion, and when we judge that an object is admirable, 
that judgment guides us toward having certain attitudes 
toward the object in question. By contrast, the value of 
correctness or appropriateness is exemplified by various 
different attitudes (including attitudes of admiration). 
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A judgment about which attitudes are correct does not 
guide us toward having any (higher-order) attitude 
toward those attitudes; it directly guides us toward hav- 
ing the lower-order attitudes themselves. On reflection, 
it is clear that judgments about rationality guide us in 
the same way. Once it is conceded that both correctness 
and rationality are distinctive in this way, it seems more 
plausible that correctness and rationality are values, 
which are connected to each other in the probabilistic 
way that I have described. 

The important point for our present purposes is that 
this approach to rationality clearly unifies both practical 
and theoretical rationality: it applies the same account— 
the idea of irrationality as bad news about correctness—to 
both practical and theoretical rationality. The difference 
between practical and theoretical rationality lies in the 
different standards of correctness that apply to belief 
and choice: the appropriate object of belief is what is true, 
and the appropriate object of choice is what is feasible and 
good. The specific features of rational belief result from 
applying this unified account of rationality to the stan- 
dard of correctness for belief, while the specific features 
of rational choice result from applying the same account 
of rationality to the standard of correctness for choice. 
If this account can be successfully defended, then on 
both the practical and the theoretical side, rationality 
consists in thinking in a way that provides good news 
about its own correctness. 


Notes 


1. Aristotle seems sometimes to use “praxis” in a broader sense 
in which it includes all voluntary behavior (1111a26). But in 
this broader sense, action does not require “reason” (since 
even nonhuman animals and children are capable of praxis in 
this sense). So I shall focus on the narrower usage of the term 
here. 


2. This translation is from Aristotle (1999). 
3. These translations are from Kant (1992, 2000, 2012). 


4. For a different conception of “rational requirements,” see 
Broome (2013, chapter 7). 


5. For an illuminating discussion of the “Dutch book” and 
related arguments, see Hajek (2005). 


6. Millgram (1995) has powerfully argued that Hume himself 
was a nihilist about practical rationality—that is, he denied that 
terms like “rational” can strictly and literally be used to evalu- 
ate actions and volitions at all. 


7. For a clear recent example of this approach, see Steglich- 
Petersen (2018). 


Ralph Wedgwood 


8. One prominent philosopher who gives an account of ratio- 
nality in terms of what one has sufficient “apparent reason” to 
do is Parfit (2011, pp. 34-35). 


9. For a more detailed account of this values-centered approach, 
see Wedgwood (2017). 
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2.3 Mental Models, Reasoning, and Rationality 


P. N. Johnson-Laird 


Summary 


Is human reasoning rational? This chapter argues that it 
is not based on logic or the probability calculus. Instead, 
rational inferences aim for accurate models of the world. 
An intuitive system uses mental models that ignore what is 
false in possibilities. So, it succumbs to systematic fallacies. 
But a deliberative system relies on explicit models that also 
represent what is false in possibilities, and so it can cor- 
rect the fallacies. Neither system knows how to infer the 
probability of compound assertions, such as conjunctions. 
So, inferences often violate the probability calculus. In 
tests of inductive hypotheses, intuitions aim to establish 
examples of them; with feedback from tests, deliberations 
aim also to show no counterexamples exist. Reasoners can 
abduce algorithms and explanations that resolve inconsis- 
tencies between facts and inferences. The biggest risk in all 
reasoning is to overlook a possibility, but a simple proce- 
dure helps to prevent this sort of irrationality. 


Science can never grapple with the irrational. 
—Oscar Wilde, An Ideal Husband 


1. Rationality and Why It Is Not Based on Logic 


Biden will impose new tariffs on Russia or China, or 
both of them. 


So, it is possible that he will impose new tariffs on Russia. 


People make this inference, and it seems sensible. But, 
as you will soon see, it violates any logic dealing with 
possibilities. The inference raises the questions of what 
counts as rational in reasoning and to what extent 
naive individuals—those untrained in logic or cognate 
disciplines—meet this target. 

Certain beliefs, inferences, and actions are irrational. 
For instance, it is irrational to believe that the Earth is flat 
(Sperber, 1995), to infer that the bow doors of a car ferry 
are closed because no one has reported otherwise (Yard- 
ley, 2014), or, like the man who kissed his pet, to fail to 


think of an obvious possibility (his pet was a rattlesnake; 
see Northcutt, 2002). Yet, theorists disagree about what 
counts as rational thinking (see section 1 of this volume). 
It could be, for example, any mental process that is 


* conducive to the survival of your genes and those of 
your relatives, 


* predisposed to yield truth, and 


* based on logic, the probability calculus, decision the- 
ory, or game theory. 


The unqualified use of “logic” here and throughout this 
chapter refers to classical logic (see chapter 3.1 by Stein- 
berger, this handbook). 

It is hard to resolve the theorists’ disagreement, 
because the only satisfactory way to do so should itself 
be rational. And, alas, that is what’s at stake. This diffi- 
culty has sometimes led psychologists to describe infer- 
ences rather than to evaluate them (cf. chapter 1.2 by 
Evans, in this handbook). But if psychologists want to 
improve reasoning, they cannot ignore criteria for ratio- 
nality. So, what are they? 

Once upon a time, the answer was simple. It was: rea- 
soning based on the laws of thought or, as Boole (1854) 
remarked, the laws of “right reasoning,” which he for- 
malized in a single algebra for both logic and probability. 
It should yield valid inferences, which are those whose 
conclusions are true in every case in which all their 
premises are true (Jeffrey, 1981, p. 1). It also requires that 
there is at least one case in which all the premises are 
true. Otherwise, contradictions imply any conclusion 
whatsoever, and naive individuals do not make such 
inferences. The preceding definition of validity is inde- 
pendent of logic. It hinges on the meanings of assertions, 
as the following inference illustrates: 


Necessarily, the plants will die. 
Therefore, the plants will die. 
It is valid in one logic of possibilities, but not in another, 


because the meaning of “necessarily” differs between the 
two logics (Girle, 2009). 
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Logic depends on formal rules that yield proofs of 
valid inferences, and psychologists once took for granted 
that logic underlies everyday reasoning (see chapter 3.2 
by O’Brien, this handbook). But an immediate problem 
is that logic concerns the relations between sentences in 
a formal language, whereas inferences in daily life con- 
cern the propositions that utterances in natural language 
convey in context. To apply a formal rule of inference, 
such as A or B or both; not A; therefore, B, to propositions, 
you need to know their “logical forms.” No algorithm 
yet exists that can carry out this task; it is extraordi- 
narily difficult (see Keene, 1992). Consider, for example, 
the disjunctive premise in our opening example: Biden 
will impose new tariffs on Russia or China, or both of them. 
You understand that “or China” abbreviates the clause, 
“Biden will impose new tariffs on China.” So, the logical 
form of the premise matches A or B or both in the formal 
rule above. This understanding depends on knowledge. 
And once you take that into account, there is no need 
for logical form. As we’ll see, reasoning can be based on 
representations of the meaning and reference of asser- 
tions. Another difficulty is that sensible inferences in 
daily life often diverge from logic. The opening example 
calls for a logic dealing with possibilities, of which there 
are infinitely many (Girle, 2009). Yet, the inference is 
invalid in all of them (Johnson-Laird & Ragni, 2019). In 
logic, the disjunction is true given that at least one of 
its two clauses is true. But it isn’t valid to infer just one 
clause from a disjunction of two. Imagine, say, that it is 
impossible that Biden will impose new tariffs on Russia 
but true that he will impose them on China. In this case, 
the disjunctive premise is true, but the conclusion is false, 
and so the inference is invalid in logic. We see later why 
people nonetheless make it and why some psychologists 
have given up logic in favor of the probability calculus as 
the basis of reasoning (cf. section 4 of this volume). 

The present chapter advocates neither logic nor prob- 
ability as the foundation of reasoning. Instead, its starting 
point is evolution. Living creatures evolved with a per- 
ceptual apparatus that constructs internal models of the 
world. The word “model” here signifies representations 
that are finite and iconic in that their structure mirrors 
the structure of the world. These models were adaptive 
for our evolutionary ancestors, and we have inherited the 
ability to construct them. And in almost any situation, 
we have the ability to conceive of a small exhaustive set 
of possibilities. Anything common to all of them is cer- 
tain, unless they concern what is permissible—in which 
case, it is obligatory. Craik (1943) wrote that our minds 
construct small-scale models of the world to help us to 
make decisions but that reasoning depends on verbal 
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rules. The modern “model” theory assumes that reason- 
ing relies instead on iconic models, which represent pos- 
sibilities (Johnson-Laird, 1983). It postulates this working 
definition of right reasoning: 


Rational inferences yield accurate models both of possibilities 
and of their implications. 


The chapter overlooks strategic reasoning in games (see 
section 9 of this volume); as the late Reinhard Selten once 
remarked, “Game theory is rational theology” (Steingold 
& Johnson-Laird, 2002). Otherwise, it deals with all the 
main sorts of reasoning: deductive, probabilistic, induc- 
tive, and abductive. 


2. Deduction and Possibilities 


2.1 The Theory of Mental Models 

The theory postulates a mechanism that builds models 
from the premises of inferences, which can be perceptions, 
descriptions, knowledge, or beliefs. So, a premise such as 


There’s a circle or a triangle, or both, in the picture 


has a set of three models of the possibilities in the 
picture: 
fo) 
A 
o 6U6UA 


Together, they have the force of an exhaustive conjunc- 
tion of three possibilities that each hold in default of 
knowledge to the contrary. And so the first model yields 
the inference 


Therefore it is possible that there is a circle in the picture. 


The inference is valid by default, and individuals make it 
(Hinterecker, Knauff, & Johnson-Laird, 2016). They also 
estimate the conclusion as less probable than that of the 
premise, which is contrary to probabilistic approaches to 
reasoning (see below). Defaults are commonplace. You 
infer that a bird flies until you discover that it is an emu. 
The model theory extends the concept to the meaning 
of connectives, such as “or.” 

Logic stipulates that a conditional, If A then B, is true 
in every case except when A is true and B is false—in this 
case alone, the conditional is false. The general claim 


If anything is a swan then it is white 
is thus equivalent in logic to 
If anything isn’t white then it isn’t a swan. 


So, a green carnation is corroboratory evidence for the 
claim: it isn’t white, and it isn’t a swan. This well-known 
“paradox” of confirmation holds for logic (Hempel, 1945). 
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In contrast, the model theory treats the conditional about 
swans as true provided that examples are possible (swans 
that are white), but counterexamples are impossible 
(swans that are not white). Things that are not swans are 
possible whether the conditional is true or false (Johnson- 
Laird, Khemlani, & Goodwin, 2015). So, carnations are 
irrelevant. Likewise, scientists need examples of a hypoth- 
esis that they are trying to induce or test from observa- 
tions (Nicod, 1950). Science, as many major practitioners 
have acknowledged, is a refinement of everyday thinking 
(see chapter 14.1 by Andersen & Andersen, this hand- 
book). And a famous test illustrates scientific practice. The 
general theory of relativity predicts that if a ray of light 
passes close to a heavenly body, it should deviate toward 
the latter by a precise amount depending on its distance 
from the heavenly body’s center (Einstein, 1920/2004, 
p. 111). To test this hypothesis, astronomers measured 
the apparent positions of “fixed” stars when light from 
them passed close to the sun—a situation satisfying the 
if-clause of the hypothesis—to determine whether exam- 
ples of the hypothesis occurred and counterexamples did 
not. The results corroborated general relativity. 

Reasoning based on models can deliver valid inferences, 
and validity is part of right reasoning. But it is not enough. 
The validity of the following inference is undeniable: 


It’s raining. 
Therefore, it’s raining and it’s raining. 


Yet, it is useless, except as an illustration of logic. In 
rational inferences, individuals should draw conclusions 
that are parsimonious (not just conjunctions of prem- 
ises), that are novel (not just a repetition of a premise), 
and that maintain the semantic information in the 
premises, which guarantees validity (Johnson-Laird & 
Byrne, 1991). These are all emergent properties of infer- 
ences based on models. One notable consequence occurs 
with premises such as 


None of the teachers is poor. 


None of the parents is poor. 


Most people respond, “Nothing follows.” The premises 
establish no definite relation that meets the constraints 
above (see chapter 3.3 by Khemlani, this handbook). 
But the response is contrary to logic, which allows that 
infinitely many valid conclusions follow from any set of 
premises. They include some that people might accept 
if they were presented to them (e.g., “It is possible that 
some of the teachers are parents”). And many deductions 
are downright stupid. The model theory distinguishes 
those that reasoners draw for themselves within those 
that are valid. 
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2.2 Irrational Deductions 

Do naive individuals reason rationally? Even Boole 
(1854) recognized that people err, and so the nature of 
their mistakes matters. Haphazard errors don’t threaten 
rationality. They fail to reflect underlying competence. 
But the model theory predicts systematic fallacies. It 
assumes that intuitive inferences rely on a simpler system 
of reasoning than deliberations: it postulates two systems 
of reasoning, which can lead to different conclusions. 
Wason was the first to formulate such a dual-system the- 
ory, but unlike other theories (see chapter 1.2 by Evans 
and chapter 2.5 by Klauer, both in this handbook), both 
its intuitive and deliberative systems were described in an 
algorithm (Johnson-Laird & Wason, 1970a). 

The crux for rationality is whether reasoners rely on 
mental models or on explicit models. As earlier, the set of 
mental models for the disjunction “There’s a circle or a 
triangle, or both” is: 

fo) 

A 


° A 


The first model represents the possibility that there is a 
circle but nothing about triangles. Mental models in the 
intuitive system represent only what is true in each pos- 
sibility. In contrast, explicit models in the deliberative 
system use negation to represent clauses in the premises 
that are false in a possibility. The explicit models of the 
preceding disjunction are therefore: 


o AA 
<0 A 
o «UA 


where “=” denotes the mental symbol for negation, 
which is linked to its semantics. 

The fallacies from mental models are ubiquitous 
(Khemlani & Johnson-Laird, 2017). Here is an example 
from two exclusive disjunctions, that is, those in which 
not both of their clauses can hold: 


Either the circle is in the picture or else the triangle is. 


Either the circle is in the picture or else the triangle isn’t. 


Could both of these assertions be true at the same time? 
Most people say, “Yes.” They rely on mental models and 
grasp that the circle is possible according to each prem- 
ise. They overlook that when one clause of an exclu- 
sive disjunction is true, the other clause is false. So, the 
explicit models of the two premises tell a different story. 
The first premise has the explicit models 


o A 
—0 A 
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And the second premise has the explicit models 


o UA 
=o =A 


The two sets have no possibility in common. And so 
the two disjunctions cannot both be true. Mental mod- 
els lead to predictable fallacies, as in this example, but 
explicit models can correct them—in principle, they 
yield only deductions that are valid according to the 
model theory. They demand more capacity in working 
memory, and an inference can exceed that capacity—a 
factor that differs from one person to another. Deduc- 
tions based on “or” and other connectives are computa- 
tionally intractable—the required processing for many 
distinct premises can exceed the capacity of a finite 
brain, even one as big as the universe (see the Overview 
by Knauff & Spohn, this handbook). 

The model theory applies to all the main domains of 
deductive reasoning (see chapter 3.3 by Khemlani on 
syllogisms, chapter 6.3 by Byrne & Espino on counter- 
factual reasoning, chapter 13.2 by Ragni on spatial and 
temporal relations, and chapter 13.3 by Knauff on visu- 
alization, all in this handbook). And its intuitive and 
deliberative processes have been implemented in a com- 
puter program for reasoning with connectives (mSen- 
tential, available at https://mentalmodels.princeton.edu 
/models/). It takes as input premises in natural language 
and evaluates the validity of a putative conclusion as 
necessary, possible, or impossible. From simple premises, 
it can construct its own conclusion in keeping with the 
constraints on rationality described earlier. It embodies 
the idea that individuals can make valid deductions, but 
in certain cases, they succumb to illusory inferences. 
They are a consequence of mental models, which rep- 
resent only what is true in possibilities. This focus may 
reflect the perceptual origins of models. 


3. Probability 


3.1. Probabilistic Theories of Reasoning 

The probability calculus is nothing more than common 
sense according to Laplace (1951/1820). This view is 
embodied in psychological theories that replace logic with 
probabilities as the foundation of everyday reasoning (see 
chapter 4.4 by Pfeifer, chapter 4.5 by Chater & Oaksford, 
and chapter 4.6 by Oberauer & Pessach, all in this hand- 
book). On this “probabilist” account, the inference 


If it rained then it was cold 
It rained 


Therefore, it was cold 
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depends neither on logical rules nor on mental models 
but on probabilities. These theories hark back to a treat- 
ment of probabilistic validity (p-validity) applying to 
conditionals (Adams, 1998). In its simplest definition, an 
inference is p-valid if its conclusion is no less probable 
than its premises in any consistent assignment of prob- 
abilities to its clauses. 

Probabilists have criticized the model theory for diverg- 
ing from orthodox logic (e.g., Oaksford, Over, & Cruz, 
2019). But, by design, the model theory differs from logic. 
And probabilism has difficulties of its own, which illumi- 
nate the nature of rationality. One difficulty is that possi- 
bilities can underlie probabilities, but probabilities cannot 
underlie possibilities. Consider a museum attendant who 
gives this permission to a visitor: 


If you have a ticket you can enter now or in fifteen 
minutes. 


Knowing that she has a ticket, the visitor infers: 
I can enter now. 


The inference is sensible, and the model theory pre- 
dicts it. Yet, it violates logic to infer one clause from a 
disjunction, and the inference is also p-invalid. But, as 
probabilists can point out, probabilities don’t apply to 
permissions. However, the same sort of inference occurs 
with descriptions: 


If the weather is bad, then it’s possible that there’s flood- 
ing or a tornado. 


The weather is bad. 
So, it’s possible that there’s flooding. 


The model theory assigns probabilities to possibilities, 
and other methods exist to do so too (e.g., Lewis, 1981). 
And the preceding inference creates a dilemma for prob- 
abilism: either a probabilistic account does not apply to 
it, or else it violates p-validity. 

How do naive individuals infer probabilities? The 
answer is: with difficulty. And probabilist theories have so 
far failed to explain the process. Consider two assertions: 


A: Trump is the Republican presidential candidate in 2024. 
B: Warren is the Democratic presidential candidate in 


2024. 


Given such assertions, participants are happy to make 
estimates of the following triples of probabilities (Khem- 
lani, Lotstein, & Johnson-Laird, 2015): 

* P(A), P(B), and P(A and B). 

* P(A), P(B), and P(A or B or both). 

* P(A), P(B), and P(B given A). 
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Each set fixes the complete joint probability distribution 
(JPD) for A and B i.e., the probabilities of each of these 
four exhaustive cases: A&B, A & not-B, not-A &B, and 
not-A & not-B. Estimates of the triples above showed 
that the participants making them didn’t know how to 
compute probabilities. Their estimates yielded robust 
violations of the principle that the percentage prob- 
abilities in the JPD should sum to 100%. Likewise, indi- 
viduals do not infer numerical probabilities unless the 
premises or the task prompts them to do so (Goodwin, 
2014), which is just as well, given their incompetence 
with numerical probabilities. 


3.2 The Model Theory of Probabilities 
The model theory provides a mechanism for computing 
probabilities. Given this problem: 


* There is a box in which there is a yellow card, or a 
brown card, or both. 


* What is the probability that in the box there is a yel- 
low card and a brown card? 


individuals tend to respond with estimates of around 
33%. Such “extensional” inferences follow from the pro- 
portion of models of possibilities in which the outcome 
occurs, and the disjunction yields three possibilities, of 
which only one satisfies the case in question (Johnson- 
Laird, Legrenzi, Girotto, Legrenzi, & Caverni, 1999). 
Similar models with numerical tags allow individuals to 
infer answers to problems of the following sort (Girotto 
& Gonzalez, 2002), in which the arithmetic is simple 
and calls for separate estimates of the denominator and 
the numerator of a probability: 


The chances that Pat has the disease are 4 out of 10. If she has 
the disease, then the chances are 3 out of 4 that she has the 
symptom. If she does not have the disease, then the chances 
are 2 out of 6 that she has the symptom. Pat has the symp- 
tom. Pat has __ chances of having the symptom, and among 
them she has __ chances of having the disease. 


As the example shows, frequencies from observations 
are not necessary for correct inferences. The participants 
can envisage the solution in a series of updates to mod- 
els of Pat’s chances. The initial model establishes that 
Pat has 4 chances of having the disease and 6 chances of 
not having the disease. Likewise, her chances of having 
the disease divide into 3 chances in which she has the 
symptom and 1 chance in which she does not. And her 
chances of not having the disease divide into 2 chances 
in which she has the symptom and 4 chances in which 
she does not. In sum, the models of Pat’s chances are as 
follows: 
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disease symptom: 3 
disease - symptom: 1 
— disease symptom: 2 
- disease -—symptom: 4 


So, Pat has 5 chances (out of 10) of having the symp- 
tom, and given the symptom, she has 3 chances out of 5 
of having the disease. So, with simple arithmetic, naive 
individuals can use this “subset principle” (rather than 
Bayes’ theorem) to estimate a conditional probability. 

Many events in daily life have no frequencies of occur- 
rence (e.g., the event that Trump will be reelected). Yet 
people in such cases can infer numerical probabilities. 
The mystery is where the numbers come from. The model 
theory offers a solution: they come from the proportions 
of models, not of the event itself, but of its occurrence 
in models of relevant evidence (Khemlani et al., 2015). 
So, for Trump’s reelection, relevant evidence is, for exam- 
ple, that most U.S. presidents running for a second term 
haven't been reelected. This yields a model in which the 
smaller proportion of possibilities supports reelection. 
The model theory postulates that the intuitive system rep- 
resents this magnitude in a nonnumerical way. It relies on 
an icon akin to one that nonnumerate individuals use to 
represent magnitudes (see, e.g., Carey, 2009). The greater 
the length of the icon, the greater the magnitude that it 
represents, so a probability is represented as a pointer on a 
“scale” extending from an origin representing impossibil- 
ity to an end point representing certainty: 


I--- | 
Subsequent evidence can shift the pointer one way or 
another, and its end point yields intuitive judgments, 
such as, “it’s unlikely.” The deliberative system can trans- 
form the icon into a numerical estimate: the probability 
is 40%. These two systems implemented in a computer 
program yield corroborated predictions about how indi- 
viduals estimate the probabilities of assertions, includ- 
ing their propensity to make estimates that violate the 
probability calculus (Khemlani et al., 2015). The origins 
of these estimates, which go back to infancy, are in sam- 
pling from models of the world (e.g., Téglas et al., 2011). 

Inferences of numerical probabilities should not flout 
the probability calculus. If they do, those who make 
them are vulnerable in principle to a “Dutch book”—a 
series of bets in which they are bound to lose money. 
Most of us, alas, often make irrational estimates of prob- 
abilities. Only rare experts are rational. A question that 
tests expertise in probabilities is about two events, A and 
B: what three sorts of estimate akin to the triples in sec- 
tion 3.1 fix the JPD in a way that always sums to 100% 
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whatever their numerical values? A correct answer is 
given below, but meanwhile readers should remember 
that the question stumps almost everyone. 

Granted that most of us are not experts, our rational- 
ity can be assessed from our performance with simple 
extensional problems. But even in these tasks, we are 
vulnerable to illusory inferences: 


There is a box in which there is at least a red marble, or 
else there is a green marble and there is a blue marble 
in the box, but not all three marbles. 


What’s the probability that there is a red marble and a 
blue marble in the box? 


Most people say their joint occurrence is impossible, 
and so the probability is zero. Their estimate is predicted 
from the two mental models of the premise: 


red marble 


green marble blue marble 


But these models fail to take into account the cases in 
which it is true that there is a red marble but false that 
there is both a green and blue marble. The latter conjunc- 
tion is false, for instance, in case there is only a blue mar- 
ble. So, there can be both a red marble and a blue marble 
in the box, and the zero estimate is an illusion. 

In sum, probabilities are tricky. Their calculus is not 
intrinsic to human mentality. That is why it was necessary 
to invent it. As a philosopher once remarked, “Anyone 
who played dice in Roman times armed with the prob- 
ability calculus would soon have won the whole of Gaul” 
(Hacking, 1975). Ignorance can underlie irrationality. 


4. Induction and the Testing of Hypotheses 


4.1 Constraints on Induction 

Much of our reasoning is outside the scope of logic: 
it is inductive rather than deductive. And, as Hume 
(1748/1988, section IV) argued, induction relies on habit, 
not necessity. Theorists sometimes propose formal rules 
for induction, but they inherit the problem of logical 
form, and knowledge is crucial. If the first 10 people that 
you meet on a desert island are overweight men, knowl- 
edge guides you to induce that its inhabitants are more 
likely to be all obese than all men. 

Knowledge can modulate the interpretation of con- 
nectives (e.g., Quelhas & Johnson-Laird, 2017), and so 
the boundary between deduction and induction is often 
unclear. The following inference is valid: 


If it rained then the plants didn’t die. 
In fact, the plants did die. 


So, it didn’t rain. 
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Here’s another example of what looks like the same sort 
of premises: 


If it rained then it didn’t pour. 
In fact, it did pour. 


Yet, you would be irrational to infer that it didn’t rain. 
You know that in a meteorological context, it pours 
means that it rained hard. This knowledge modulates 
the interpretation of the conditional so that it refers 
only to a conjunction of two default possibilities: it 
rained but didn’t pour, and it didn’t rain. So the second 
premise contradicts the first premise. The example illus- 
trates again the difficulty of pinning down logical form. 

Beliefs underlie many inductions, which are therefore 
bound to be dangerous. Theorists have developed sys- 
tems for updating beliefs (see section 5 of this volume). 
The trouble is that a set of beliefs can be consistent but 
irrational, because they do not correspond to reality. 
As a commentator on accidents at sea wrote, “Despite 
the increasingly sophisticated equipment, captains 
still inexplicably turn at the last minute and ram each 
other ... they built perfectly reasonable mental models 
of the world, which work almost all the time, but occa- 
sionally turn out to be almost an inversion of what really 
exists” (Perrow, 1984, p. 230). 

After a month, a new sort of light bulb fails in your 
living room. You draw an analogy: the bulbs are like 
some new flawed computer chips. You formulate an 
explanation: the bulbs’ filaments contain an impurity. 
These processes are inductive, and they play a part in 
science and technology. For example, epidemiology 
was founded in part on John Snow’s discovery of how 
cholera was communicated from one person to another. 
His discovery began with inductions: cholera is a single 
specific disease, its spread follows the trade routes—an 
infected mariner arriving in a port spreads the disease 
to those in contact with him. More puzzling was that 
it could leap over considerable distances. Snow inferred 
that “particles” of the disease in sewage contaminated 
drinking water. He was right: the particles were bacteria. 
The Wright brothers’ invention of the airplane rested 
on numerous inductions. Maxim, the inventor of the 
eponymous gun, said, “Give us a motor, and we will 
very soon give you a successful flying machine.” Wil- 
bur Wright countered with an induction from the fate 
of glider pilots: if the engine fails and the pilot has no 
control of the aircraft, then it will crash and the pilot 
could be killed. The brothers’ first goal was therefore 
to discover how to control a glider rather than to build 
a motor (for Snow’s and the Wrights’ inductions and 
deductions, see Johnson-Laird, 2006). 


Mental Models, Reasoning, and Rationality 


One constraint on inductions is similarity: similar 
causes have similar effects (Hume, 1748/1988, p. 80; 
Tversky & Kahneman, 1983). Another constraint is that 
physical contact implies causation—it is one cause of 
cholera’s transmission. Yet, these constraints are those 
on which magical thinking and superstitions rely, and 
all cultures are susceptible: similarity yields homeopathic 
magic (take an antibiotic for a sore throat), and contact 
yields contagious magic (don’t touch people with AIDS). 
There are too many possible hypotheses, and so any 
constraints are better than none. The only security in an 
induction comes from tests of its conclusion. 


4.2 Testing Hypotheses 

A test of an induction needs to determine whether its 
consequence is true or false. The most influential experi- 
mental paradigm for such studies is Wason’s (1968) 
selection task. The experimenter chooses four cards from 
a pack in which, as the participants know, all the cards 
have a letter on one side and a number on the other 
side. They are laid out in front of the participant: E F 2 3. 
The participant’s task is to select only those cards that, 
if turned over, would show whether a hypothesis about 
the four cards is true or false: 


If there is an “E” on one side of a card then there is a “2” 
on the other side. 


With an abstract hypothesis of this sort, participants 
tend to select the E and 2 cards or the E card alone. 
The selection of the 2 card is pointless, because nothing 
on its other side can refute the hypothesis. But what 
stunned psychologists and launched hundreds of stud- 
ies was the participants’ failure to select the 3 card. If it 
had an E on its other side, the hypothesis above would 
be false. Wason, his colleagues, and many others took 
this oversight to be irrational. Not everyone agreed. 
One view was that the probabilities of events related 
to everyday conditionals made it rational not to select 
the 3 card (see chapter 4.5 by Chater & Oaksford, this 
handbook). In contrast, the model theory’s algorithm 
(Johnson-Laird & Wason, 1970a) predicts that the test of 
a general hypothesis needs to establish the possibility of 
examples and the impossibility of counterexamples (see 
section 2.1). So, selections of cards should be dependent 
on one another (e.g., the 2 card should tend to be selected 
with the E card, as an example of the hypothesis). It also 
predicts that the task and its contents should affect the 
likelihood that participants select potential counterex- 
amples. A meta-analysis of over 200 experiments corrob- 
orated these predictions (Ragni, Kola, & Johnson-Laird, 
2018). 
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In a repeated version of the selection task, partici- 
pants had to make an efficient test of the generaliza- 
tion “All the triangles are white” about the shapes in 
two boxes (Johnson-Laird & Wason, 1970b). As the 
participants knew, one box contained 15 white shapes 
and the other box contained 15 black shapes. On each 
trial, they selected a shape from one of the two boxes 
to see whether or not it was a triangle. The participants 
started by selecting white shapes—potential examples 
of the hypothesis—but sooner or later switched to black 
shapes—potential counterexamples—and examined 
them all. The original selection task fooled them: they 
had just one opportunity to make a rational selection. In 
the repeated task, sooner or later they had the insight to 
select all potential counterexamples. 


5. Abduction, Explanations, and Nonmonotonicity 


5.1 Abductions That Resolve Inconsistencies 

The author and a friend were sitting BC (before cell phones 
and corona virus) outside a café in Provence. We were 
waiting for two other friends, and we knew two things: 


They had gone to pick up the car. 
And if so then they would be back within 10 minutes. 


So, we thought, they’ll be back in 10 minutes. When 
they hadn’t returned after 20 minutes, we realized that 
our conclusion was false. In logic, such a contradiction 
implies any conclusion whatsoever. Logic is “mono- 
tonic”: a fact that contradicts an earlier valid inference 
does not call for its retraction. In daily life, however, it is 
rational to withdraw conclusions that facts refute, and so 
reasoning is “nonmonotonic” or “defeasible” (see chap- 
ter 5.4 by Gazzo Castafieda & Knauff, this handbook). 
William James (1907) wrote, “The new fact preserves the 
older stock of truths with a minimum of modification, 
stretching them just enough to make them admit the 
novelty” (p. 59). And this “minimalist” view has many 
defenders (see chapter 5.2 by Rott, this handbook). Yet, 
outside the café, we were concerned to figure out what 
had happened to our friends. We thought of various 
explanations: they had got lost (improbable), the police 
had stopped them (improbable), they had been in an 
accident (improbable), and they couldn’t start the car 
(possible, because it had happened before). So, we rea- 
soned, we had better sit tight and wait, because if we 
went in search of them and they returned to the café, 
they wouldn’t know where we were. Sure enough, soon 
afterward, the car came spluttering into view, and we 
hopped in while its engine was running. It had needed 
a tow to start. 
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This anecdote exemplifies the model theory. Any 
inference in daily life is made in default of knowledge 
to the contrary. That is, conclusions can turn out to be 
false. In such cases, the primary task is to create an expla- 
nation that resolves the inconsistency between the fact 
and the premises. It can then guide the decision about 
what to do. 

The creation of an explanation is known as “abduc- 
tion.” Its rational goal, as ever, is accuracy. The process is 
a sort of induction, because it goes beyond the premises 
and depends on knowledge. But it seeks more than an 
inductive generalization: it introduces new concepts in 
order to create an explanation. The process is triggered 
when a fact refutes an earlier inference. For example, the 
premises 


If a cobra bit her then she’ll die 
A cobra bit her 


elicit a mental model of a snake biting her and her death. 
But the fact 


She did not die 


is inconsistent with this model. And the conflict triggers 
an attempt to resolve the inconsistency. The whole non- 
monotonic process is implemented in the mSentential 
program. Its first step is to compute the facts: a cobra 
bit her but she did not die. It searches its knowledge for 
such cases and their causes. The program’s knowledge 
base contains several relevant possibilities in which a 
deadly snake bites someone who subsequently does not 
die: 

* the person takes an antidote, 


* a tourniquet blocks the poison, and 


* someone sucks out the poison. 


Human reasoners tend to assess the probability of each 
explanation. The program chooses one at random and 
then uses the same procedure again to search for its cause. 
If successful, the result is a causal chain, for example, 


A cobra bit her and she used a tourniquet and the tourni- 
quet blocked the poison and she did not die. 


It also uses the first cause in the chain to construct a 
counterfactual description of what would otherwise 
have occurred: 


If she had not used a tourniquet then she would have 
died. 


Of course, the program is itself only a partial model 
of human reasoning. When you create explanations, 
you work with real models of the world (not strings of 
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words), and you assess the relative probability of differ- 
ent putative explanations. 

The theory predicts that individuals should prefer 
causal explanations to minimal revisions. Indeed, they 
rate a causal chain as more probable than either the 
cause alone or the effect alone (Johnson-Laird, Girotto, & 
Legrenzi, 2004)—a violation of minimalism and a fallacy 
in which a conjunction is judged to be more probable 
than its constituents (see Tversky & Kahneman, 1983). 
When individuals are asked what follows from incon- 
sistent premises, they tend to explain the inconsistency. 
And they judge such explanations as more probable than 
minimal revisions to the premises that restore consistency 
(Khemlani & Johnson-Laird, 2011). When they formulate 
an explanation first, a striking phenomenon occurs. They 
find it harder to detect the inconsistency—they seem 
to have explained it away (Khemlani & Johnson-Laird, 
2012). 

A consistent set of beliefs may not correspond to real- 
ity, but an inconsistent set of beliefs cannot correspond 
to reality. The ability to detect inconsistencies is there- 
fore a hallmark of rationality. Reasoners can assess con- 
sistency by trying to find a model of all the information 
they have. Such a model shows that the information is 
consistent; otherwise, it is inconsistent. Once reason- 
ers have detected an inconsistency, they can use their 
knowledge to abduce causal models to explain the ori- 
gins of the inconsistency. It is rational to try to do so, but 
no guarantee exists that the explanation is correct. And, 
on rare occasions, people fail to find any explanation 
whatsoever for an inconsistency. Yet, they outperform 
any existing computer program in creating explana- 
tions. The crux is that they know more and are better at 
retrieving pertinent information. 


5.2 Abductions of Algorithms 

Individuals who know nothing of programming can cre- 
ate simple informal algorithms. Deductions alone won’t 
work, and probabilities are no use, either. Algorithms 
are an explanation of how to do something, and so rea- 
soners need to introduce new concepts and to organize 
them in a temporal sequence. In a word, they abduce 
algorithms. For example, they can do so to rearrange 
the order of cars in a train on a railway track (Khem- 
lani, Mackiewicz, Bucciarelli, & Johnson-Laird, 2013). 
The track runs from left to right on a computer screen, 
and it has one siding, which is entered from, and exited 
to, the left track. As an illustration, you might like to 
think how you would use the siding to rearrange the 
cars ABCD on the left side of the track so that they arrive 
in reverse order on the right side of the track. The more 
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moves (and the more cars in each move) that are called 
for to solve such a rearrangement, the harder the task is. 
A more substantial task, however, is to abduce an algo- 
rithm that solves such a rearrangement for a train with 
an arbitrary number of cars. 

A computer program, mAbducer, creates such algo- 
rithms, which it can express in informal English. Here, 
for instance, is the algorithm it abduced to reverse the 
order of cars: 


Move one less than the [number of] cars to the siding. 
Move one car to the right track. 
While there are more than zero cars on the siding, 
move one car to the left track, 
move one car to the right track. 
The effect of the first move on ABCD on the left track 
is shown in the following diagram in which the square 


brackets demarcate the cars on the siding, and the dash 
shows that no cars are yet on the right side of the track. 
Move one less than the cars to the siding: A[BCD] - 
The second instruction is: 
Move one car to the right track: — [BCD]A 


Next, while there are more than zero cars on the siding, 
there is a loop of two instructions: 


move one car to the left track: 
move one car to the right track: 


BICDJA 
—[CD]BA 


The loop is repeated twice until the siding is empty, and 
the result is the cars in reverse order on the right track: 


—[]DCBA 


As mAbducer illustrates, the model theory postulates 
that people who are not programmers abduce informal 
algorithms using a kinematic model that simulates a 
solution to an instance of the problem. They observe 
what happens in the simulation and transform their 
observations into an informal description of the required 
algorithm. In principle, individuals could determine the 
number of iterations of a loop of moves by solving two 
simultaneous linear equations. But, in fact, they more 
often observe the conditions in which the loop halts in 
their simulations (e.g., when the siding is empty in the 
algorithm above). The program implements both sorts 
of loop (in both a programming language and English). 

Experiments showed that most individuals fol- 
low the observational procedure—they use a so-called 
while-loop of the sort illustrated above (Khemlani et al., 
2013). Even 10-year-old children can abduce informal 
algorithms for real toy trains of a small number of cars. 
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In one experiment, they were not allowed to move the 
actual cars, and so they tended to make gestures corre- 
sponding to the moves their algorithms required. When 
they were prevented from gesturing, they were no longer 
so accurate in abducing algorithms (Bucciarelli, Mackie- 
wicz, Khemlani, & Johnson-Laird, 2016). Gestures are an 
outward sign of inward simulation, and they reduce the 
load on working memory. 

Different rearrangements vary in the difficulty of 
their abduction. For children and adults alike, it depends 
not on the number of cars that have to be moved but 
on the complexity of the algorithm. Various measures 
exist for this complexity, but a good proxy is the num- 
ber of instructions in the algorithm. A rational algorithm 
should carry out the correct moves, and it should do so 
without unnecessary ones. It is tempting to say that it 
should be minimal (i.e., it should be of the shortest possi- 
ble length in the given programming language). There is a 
problem, however: for all but the simplest algorithms, no 
way exists to prove that an algorithm is minimal (Chaitin, 
1998). 


6. Conclusions 


Rationality can seem remote, abstract, and complex. The 
message of this chapter, however, is that it matters. What 
counts as rational is a recursive question, because its cor- 
rect answer depends on rational methods. It goes beyond 
reasoning to apply to beliefs, actions, and social interac- 
tions. But, for reasoning, the view defended here is: 


Rational reasoning constructs accurate models of the world 
based on knowledge or beliefs; infers their consequences for 
what is possible, probable, or certain; and uses these models 
to formulate explanations. 


In short, right reasoning aims for truth, parsimony, and 
explanatory power. But, as the chapter showed, humans 
are fallible. They can fail to think about what's false in 
making deductions and succumb to illusory deductions. 
They can make an incorrect inference of the likelihood 
of a compound assertion, because the probability cal- 
culus is not part of their mental equipment. One test 
of expertise is my earlier question: which three sorts of 
estimate concerning two events, A and B, fix the val- 
ues of their joint probability distribution and, regardless 
of their numerical values, always yield a JPD that sums 
to 100%? One correct answer is: the probability of A, 
the conditional probability of B given A, and the con- 
ditional probability of B given not-A. My survey of psy- 
chologists showed that few of them could answer this 
question correctly. 
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Although humans are adept at induction, no con- 
straints can guarantee the truth of its outcomes. The 
heuristics that humans use are often irrational. They 
yield truths and trumpery, science and superstitions. 
One rational criterion, however, is that hypotheses 
should be open to refutation—for Popper (1959), this 
criterion separates science from nonscience. But naive 
individuals in the selection task often focus on corrobo- 
ratory examples of a hypothesis. In its repeated version, 
however, they do realize the need to ensure that coun- 
terexamples do not exist. 

When a fact refutes the conclusion of a valid infer- 
ence, logic tells us nothing—it licenses any conclusion 
whatsoever from a contradiction. In contrast, individu- 
als innocent of philosophical niceties follow neither 
logic nor the advice to make a minimal revision of their 
beliefs. Instead, they search for an explanation that 
resolves the inconsistency. These explanations are not 
always minimal. The truth, as Oscar Wilde remarked, is 
seldom simple (and never pure). One domain, however, 
does call for minimalism: the abduction of algorithms. 
Like so many ideals, alas, no way exists guaranteed to 
reach the shortest possible algorithm. 

A common error in all sorts of reasoning is to over- 
look a possibility. Such failures are not inevitable, and 
the model theory suggests something that can be done 
to reduce them. You can learn to be more rational. You 
might imagine that you have to learn logic. It takes 
time—at least a semester’s course for the rudiments— 
and it fails to generalize to novel sorts of inference 
(Cheng, Holyoak, Nisbett, & Oliver, 1986). In contrast, a 
good method for improving reasoning should be quick 
to learn, efficacious, and practical. One method meets 
these goals. Its single instruction is: 


Try to construct all the possibilities consistent with the given 
information. 


With pencil and paper, you can list them in separate 
columns, adding a column for each new possibility 
and crossing out a column if a premise refutes its pos- 
sibility. It takes only a few minutes to learn this “model 
method,” which Victoria Bell devised (see Johnson-Laird, 
2006, p. 288). It speeds up reasoning and increases its 
accuracy (from 66% correct to 95% in one study). Once 
people have acquired the method, they can imagine the 
columns of possibilities and no longer need pencil and 
paper. It still works. And the method may be adaptable 
to other sorts of reasoning beyond deduction. 

To answer the two questions in my opening para- 
graph: right reasoning should yield accurate models of 
the world, and human reasoning is rational in principle 
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but often irrational in practice. Yet, humans realize their 
own shortcomings, and they are rational enough to 
develop tools that can help them. Contrary to Wilde’s 
epigraph at the head of this chapter, psychology grapples 
with the irrational and even dispels it . . . sometimes. 
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Summary 


Research on heuristics and biases has greatly influenced 
the current understanding of bounded rationality in 
psychology, behavioral economics, cultural anthropol- 
ogy, educational science, and neighboring disciplines of 
behavioral science (see chapter 1.1 by Sturm, chapter 1.2 
by Evans, chapter 2.2. by Wedgwood, and chapter 8.4. 
by Hill, all in this handbook). The first major section of 
the present chapter is devoted to Kahneman and Tver- 
sky’s seminal research program, providing a synopsis of 
the most prominent heuristics supposed to trigger vio- 
lations of logical coherence rules: availability, represen- 
tativeness, anchoring, and the simulation heuristic. A 
second major section deals with the contrast program 
advocated by Gigerenzer and colleagues, which high- 
lights the adaptive value of fast and frugal heuristics. 
The last major section provides a critical discussion of 
the insights gained from half a century of fascinating 
research on heuristics and biases, and of the neglected 
issues—such as normative standards and underlying 
cognitive mechanisms—which constitute open ques- 
tions to be resolved in future investigations of rational 
action and decision making. 


1. The Notion of Heuristics and Biases 
in Rationality Research 


Real life is replete with uncertainty. Even the most 
mundane and superficial actions and decisions involve 
assumptions about risks, probabilities, costs, and benefits, 
which are indeterminate and hard to evaluate objectively. 
Partner choice depends on social judgments of attractive- 
ness, on trust, and on the likelihood of future scenarios. 
Consumers compare food items for healthiness and tasti- 
ness. Teachers’ grading decisions entail inferences about 
students’ competence and performance. Lying must be 
discriminated from veracious statements in everyday 
communication. Health-related behavior depends on 
the assessment of risks and dangers of nutrition, hobbies, 


sports, and transportation means. And predictions of 
future outcomes—in politics, sports, economic and eco- 
logical developments, and private affairs—are virtually 
never made with certainty. 

In such an indeterminate world, there is often no 
absolute normative criterion for rational behavior. Sci- 
entifically approved answers to the question of how risks 
and costs can be minimized and how benefits and satis- 
faction can be maximized rely on rules of statistics and 
probability assessment, as a surrogate for a determinis- 
tic rule. Human judgments and decisions often do not 
rely on the mathematics of probability and statistics but 
rely instead on an alternative strategy, namely, on sim- 
plifying proxies or rules of thumb known as heuristics. 
Under specific conditions, “these heuristics are highly 
economical and usually effective, but they lead to sys- 
tematic and predictable errors” (Tversky & Kahneman, 
1974, p. 1131). 

The purpose of the present chapter is to provide an 
overview of half a century of intensive research on heu- 
ristics and biases (Gigerenzer & Gaissmaier, 2011), which 
has had a strong impact on the study of rationality in 
behavioral science in general and on contemporary work 
in psychology, economics, biology, and philosophy in 
particular. Starting from a synopsis of Daniel Kahneman 
and Amos Tversky’s groundbreaking demonstrations 
of the most prominent heuristics (Gilovich, Griffin, & 
Kahneman, 2002; Kahneman, Slovic, & Tversky, 1982; 
Tversky & Kahneman, 1974), we move on to a broader 
discussion of bounded rationality (Simon, 1982) and of 
the adaptive beauty of heuristic inferences considered 
from an alternative perspective by Gerd Gigerenzer and 
his colleagues (Gigerenzer & Goldstein, 1996; Gigeren- 
zer, Todd, & ABC Research Group, 1999). Another major 
section will then be devoted to a critical assessment of 
what insights have been gained from this extremely 
prominent field of research and what neglected issues 
remain open to be answered in future investigations. 

Let us illustrate the basic idea with an introductory dis- 
cussion of one specific example, the availability heuristic 
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(Tversky & Kahneman, 1973), which is ideally suited to 
explain the manner in which heuristics allow for accu- 
rate judgments in many everyday situations, although 
in different situations the same heuristics may lead to 
serious biases and irrational decisions. 


2. The Availability Heuristic as an Illustrative Example 


The availability heuristic (Tversky & Kahneman, 1973) 
affords a mental tool for estimating the frequency or 
probability of events. According to scientific methodol- 
ogy, normatively adequate estimations of such frequen- 
tist quantities should be inferred from proportions in 
representative samples. However, when human individ- 
uals estimate the risk (prevalence) of a disease or an acci- 
dent, or the chances of winning a game in sports or of 
solving a problem, a representative sample is hardly ever 
available. Human memory is not a data repository; we 
do not keep representative data arrays in memory from 
which random samples could be drawn to inform unbi- 
ased frequency or probability estimates. Even when we 
do sometimes keep traces of ecological events in mem- 
ory or in some external store, they cannot be expected to 
be representative of the current reality. 

Nevertheless, we do have the capacity to estimate 
word frequencies or the likelihood of, say, different 
causes of death (Combs & Slovic, 1979) or of a certain 
team to become basketball champion, even though we 
do not keep in memory word counts, sports statistics, or 
epidemiological data about lethal events. We simply rely 
on the availability heuristic, which uses the availability 
of relevant memories as a proxy for frequency and prob- 
ability estimation. The ease with which relevant memo- 
ries (of word occurrences, sports teams’ past success, 
lethal events) come to mind affords a useful heuristic 
of astounding validity. Most of the time, the likelihood 
of a target event is substantially correlated with the 
availability of relevant traces in memory. As memories 
of people dying from cancer are more available than 
memories of people dying from car accidents, which are 
more available than memories of suicide or lightning, 
heuristic estimates reflect the true ordering (cancer > car 
accidents > suicide > lightning), yielding a remarkably 
strong correlation between objective frequencies and 
the results generated by the availability heuristic. 

It should be noted in passing that nonheuristic, pur- 
portedly normative methods also offer little more than 
approximate solutions; the norm distribution required 
to estimate a specific person’s exact cancer, accident, or 
suicide risk is simply unknown. Available statistics that 
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pertain to the entire population, or to crude substrata 
thereof, may not apply to individual persons with their 
individual lifestyle, their genetic predispositions, and the 
countless resulting combinations of risk factors. So, for a 
fair comparison, heuristics should not be contrasted with 
“objectively true values” as in psychophysics experiments 
but with normative surrogate solutions that are them- 
selves debatable and often misleading and that might also 
be called “heuristic.” Yet, despite its relative usefulness, 
the availability heuristic can lead to distinct biases when 
available memories depend on distracting, extraneous 
factors. For instance, selective media reports may render 
certain infrequent causes of death (e.g., murder, lightning) 
more available than other, objectively more frequent, 
causes of death (e.g., suicide, household accidents). These 
misleading cases are the focus of most pertinent research. 


3. Synopsis of Prominent Heuristics, Their Domain, 
and Associated Biases 


Let us now move from availability to a discussion of 
other prominent heuristics, their domain, and the kinds 
of biases that result from heuristic inference schemes. 


3.1 Representativeness 

The domain of the representativeness heuristic, one of 
the first heuristics specified by Kahneman and Tversky 
(1972), is categorization, or the diagnostic judgment 
of the likelihood that an event or person belongs to a 
category or group. A typical task may provide the fol- 
lowing description of a target person: “very shy and 
withdrawn, invariably helpful, but with little interest in 
people, or in the world of reality.” When participants are 
asked to judge whether the target person is more likely 
to be a farmer or a librarian (among other options), most 
people choose librarian, because the description is more 
representative of librarians than of farmers. They largely 
ignore the categories’ base rates, that is, the fact that the 
proportion of farmers in the general population is much 
higher than the proportion of librarians. This pattern is 
obtained even when base rates are experimentally con- 
trolled (Tversky & Kahneman, 1974). 

Similar to the availability heuristic, the representative- 
ness heuristic enables accurate judgments, in this case of 
categorization, as long as the similarity of the target per- 
son description to the vocational stereotype constitutes 
a viable proxy of the actual profession. However, when 
likeness diverges from likelihood (Shweder, 1977)—that 
is, when the person sketch resembles a librarian, whereas 
farmer is much more likely—the heuristic will be misled 
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by similarity, resulting in a distinct base-rate neglect 
(Bar-Hillel, 1984; Pennycook & Thompson, 2017). 

According to the representativeness heuristic, sam- 
ples are generally and uncritically assumed to be rep- 
resentative of an underlying entity or population. This 
assumption works in two directions: first, it is assumed 
that the properties of the underlying population can 
always be validly inferred from any given sample. Sec- 
ond, even small samples drawn from a given population 
are expected to have the same properties as the popula- 
tion at large, whereas deviating samples are supposed to 
be very unlikely. To illustrate the latter point, Kahneman 
and Tversky (1972) constructed the following task: 


All families of six children in a city were surveyed. In 72 
families the exact order of births of boys [B] and girls [G] was 
GBGBBG. What is your estimate of the number of families 
surveyed in which the exact order of births was B G B B B B? 


Both sequences are equally probable assuming invariance 
of the gender on successive births. However, they differ 
markedly from the well-known population distribution 
in terms of representativeness. Because the “law of small 
numbers” (Tversky & Kahneman, 1971) suggests that 
even in a short sequence, the ratio of boys and girls is 
close to 1:1, participants judged the first sequence to be 
more likely than the second sequence. In the same vein, 
seemingly regular chunks like B B B Bor BGBGBG 
are poor representations of random events at odds of 1:1. 
Thus, series containing less regular chunks and seemingly 
less systematic content are judged to occur with a higher 
likelihood. The same reasoning mistake is at work when 
an explicit sampling task is replaced by a random process 
or “lottery”: tossing a fair coin with the possible results of 
heads (A) and tails (T). Parallel to the above example, the 
series H T H T T H will be judged to occur more frequently 
than TH T T T T, which is again a mistake. 


3.2 Anchoring and (Insufficient) Adjustment 

Quantitative judgments—for instance, of total costs for 
a project, of the time required to complete a job, or of 
the appropriate punishment in a courtroom decision— 
constitute the domain of the anchoring heuristic (Epley 
& Gilovich, 2010; Strack, Bahnik, & Mussweiler, 2016). 
Essential for this heuristic is the assumption that quan- 
titative judgments typically start from an anchor, either 
a low or a high anchor, which is then adjusted in the 
light of relevant information. However, this adjust- 
ment process is typically insufficient or incomplete so 
that the resulting judgment remains biased toward the 
initial anchor. The anchoring heuristic thus produces 
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underestimation if the initial anchor is low and overes- 
timation if the initial anchor is high. For example, the 
overall costs for a holiday trip tend to be underestimated 
if calculation starts from zero, or from the cheapest base 
price, to which predictable costs are added. In contrast, 
costs tend to be overestimated if an overly high anchor 
is used as a starting point and all nonapplicable costs are 
subtracted. 

For a nice empirical illustration, the starting value of 
an auction often provides a good predictor of the final 
price (Ritov, 1996; but see Galinsky, Ku, & Mussweiler, 
2009). An intriguing finding from many pertinent exper- 
iments says that exposure to fully irrelevant numerical 
stimuli—such as somebody’s Social Security number— 
can cause an anchoring bias in judgments of completely 
unrelated quantities (Oppenheimer, LeBoeuf, & Brewer, 
2008; Wilson, Houston, Etling, & Brekke, 1996). In a sem- 
inal demonstration by Tversky and Kahneman (1974), 
estimates of the number of African nations in the United 
Nations were biased toward random numbers generated 
by a wheel of fortune. 

As in the case of the other heuristics, anchoring and 
(incomplete) adjustment will often result in rather accu- 
rate estimates. However, obviously, quantitative estima- 
tion can be biased to the extent that the starting anchors 
misrepresent the true value of the estimated quantity. 


3.3. The Simulation Heuristic 

The simulation heuristic is sensitive to expectancies 
resulting from mental simulation. What is easy to imagine 
or simulate mentally (e.g., one’s favorite team winning or 
losing an important match) will determine not only one’s 
expectations but also one’s emotions of regret and disap- 
pointment elicited by unexpected outcomes. For instance, 
one is particularly disappointed and frustrated when one 
misses a train by a minute rather than by 30 minutes, 
because the counterfactual simulation of still reaching the 
train on time is so vivid in the former case. As a conse- 
quence, silver-medal winners in Olympic games can be 
less satisfied and more disappointed than bronze-medal 
winners, because counterfactual mental simulations 
may compare silver medals with a missed gold medal 
but bronze medals with a worse outcome of receiving no 
medal at all (Medvec, Madey, & Gilovich, 1995). 


3.4 Availability 

As introduced at the outset, the domain of the avail- 
ability heuristic encompasses judgments of uncertain 
frequencies or probabilities. The ease with which mem- 
ory content comes to mind affords a useful proxy for 
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frequency estimation, although it can lead to distinct 
biases when ease of memory retrieval itself is biased, for 
instance, due to selective media coverage (Pachur, Hert- 
wig, & Steinmann, 2012; Reber & Zupanek, 2002). 


4. Lopsided Focus on Heuristics as Erroneous 
Inference Tools 


The greatest part of the published literature, to be sure, is 
concerned with empirical demonstrations of biases and 
erroneous heuristic inferences; relatively little research 
has assessed how often heuristic judgments are valid and 
unbiased. For example, in one of the most frequently 
cited demonstrations of the availability heuristic, Tver- 
sky and Kahneman (1973) showed that the number of 
English words with the letter “k” in the first position 
was erroneously judged to be higher than the number of 
words with a “k” in the third position. This is apparently 
due to the fact that it is easier to retrieve words with a 
particular letter in the first than in the third position. 
However, as SedImeier, Hertwig, and Gigerenzer (1998) 
have shown, this finding does not generalize to all let- 
ters of the alphabet; it is peculiar to the few letters on 
which the original demonstration had concentrated. 

Because of this lopsided research focus on the biasing 
potential of heuristics and the relative neglect of their eco- 
logical validity, accuracy, and adaptive value, heuristics are 
associated with negative connotations in the published 
literature. References to “heuristic inferences” often char- 
acterize them as sloppy, superficial, inaccurate, and irra- 
tional. The credo of many dual-process theories (Chaiken 
& Trope, 1999; Sloman, 1996; Smith & DeCoster, 2000) is 
that the remedy to cognitive biases and distortions is to 
switch from a heuristic to a systematic processing mode. 
Just as the alleged inferiority of heuristic processing is 
rarely demonstrated in a representative design, the alleged 
superiority of systematic processing modes is hardly ever 
proven. 

Given the apparent analogy to the classic perceptual 
illusions of Gestalt psychologists, the negative image of 
biases or cognitive illusions resulting from distinct heu- 
ristics (such as availability) is hard to understand. Per- 
ceptual illusions (such as the contrast effect underlying 
the Ebbinghaus illusion’) had typically been understood 
as rare failures of adaptive and strikingly efficient pro- 
cessing. In the realm of cognitive illusions, though, most 
researchers inspired by Tversky and Kahneman’s (1974) 
heuristics-and-biases approach were not interested in 
adaptive functions but almost exclusively in demon- 
strating cognitive limitations as causal origins of irratio- 
nal behavior—even in the absence of cogent evidence.” 


Klaus Fiedler, Johannes Prager, and Linda McCaughey 


The aforementioned findings on availability effects 
in estimating causes of death (Combs & Slovic, 1979) 
provide testimony for this lopsided tendency. The same 
researchers who, for example, pointed out biased newspa- 
per coverage as a plausible environmental cause of overes- 
timated murder rates (relative to suicide rates) continued 
to treat these findings as evidence for an alleged cogni- 
tive bias in selective retrieval. Neither Combs and Slovic 
(1979) nor many other authors who drew on their find- 
ings ever made an attempt to rule out the possibility that 
(environmental) biases in newspaper reporting alone may 
account for the biased estimates even when retrieval pro- 
cesses are completely unbiased. Most researchers largely 
neglected the obvious fact that heuristics may serve a gen- 
erally adaptive function, with only a few specific types of 
situations leading to the biases investigated so feverously 
by the research community. 


5. Heuristics as Indispensable Modules 
of Adaptive Intelligence 


We have already noted that for many real-life problems, 
there are hardly any purely logical, normative solutions. 
That is, so-called normative models of reality also rely 
on heuristics or proxies used to estimate the costs of a 
construction project, the severity of climate change, the 
utility of different ecological policies, or the number of 
participants required in an experiment. Indeed, heu- 
ristics are part and parcel of expert systems and of the 
most intelligent systems of adaptive cognition, as illus- 
trated in the early work of Egon Brunswik. Long before 
Kahneman and Tversky unleashed this wave of research 
into the inadequacies of human judgment by cause of 
heuristics, Brunswik’s (1955) probabilistic functionalism 
outlined a very different perspective on heuristics as 
indispensable modules of adaptive regulation. 
According to Brunswik’s lens model, the distal entities 
that are the focus of most judgments in real life—entities 
such as risk, danger, honesty, attraction, student ability, 
or interpersonal trust—are not amenable to direct per- 
ception through specialized sense organs. We somehow 
have to construe those distal entities from sets of proxi- 
mal cues, which bear only weak or modest correlations 
to the distal entities. The inferential utilization of insuf- 
ficient but useful and indispensable cues is nothing else 
but highly functional heuristic processing. For instance, 
to assess the distal variable of honesty in lie detec- 
tion, we have to rely on such imperfect heuristic cues 
as speech hesitation, gaze aversion, pitch of the voice, 
or disguised smiling (Hartwig & Bond, 2011, 2014), all 
of which are rather low in diagnosticity. Although all 
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Figure 2.4.1 


Brunswik’s (1955) lens model applied to subjective judgments of danger, mediated by five proximal cues framed to resemble some 


of the most well-known heuristics (recognition, availability, simulation heuristic). Statistical relations of proximal cues to distal 


entities are called ecological validities. The weights given to cues in subjective judgments are referred to as cue utilization. 


these cues provide only weak predictors of the distal 
criterion, honesty, they nevertheless allow for above- 
chance lie detection when an increasing number of cues 
of restricted validity are combined. Crucially, there is no 
alternative to relying on heuristic cues for lie detection, 
which is of utmost importance for adaptive behavior. 

As we have learned from Brunswik (1955), we do not 
possess sense organs even for the literal perception of 
many obviously physical attributes, such as distance. 
Rather, in-depth perception organisms have to infer 
the perceived distance of an object using such proxi- 
mal cues as disparity of the retinal images, movement 
parallax, surface texture, or blurredness of contours. All 
these cues afford only imperfect correlates of distance; 
in other words, they allow us to approximate the dis- 
tal entity in question in a crude heuristic manner only. 
Moreover, the notion of vicarious functioning highlights 
the substitutability of all these cues, none of which is 
necessary or essential for depth perception. Still, they are 
functional because their ecological validity is better than 
chance and, as mentioned before, there is no alternative 
to relying on heuristic cues. 


5.1 Using the Lens Model to Describe Heuristics’ 
Adaptive Functions 

By analogy, Hammond, Stewart, Brehmer, and Stein- 
mann (1986) used the lens model in their social judg- 
ment theory to analyze the cues used as proxies for such 
distal entities as risk, attraction, trust, or required effort 
expenditure (see figure 2.4.1). Obviously, availability and 
other heuristics afford reasonable cues, the validity of 


which is an open empirical question. However, as already 
mentioned, the validity of any alternative device supposed 
to be normatively appropriate is also an open empirical 
question. In any case, the lens model in figure 2.4.1, in 
analogy with social judgment theory (Hammond et al., 
1986), suggests a framework within which the availabil- 
ity heuristic (along with other heuristic cues) serves an 
uncontested adaptive function—namely, to allow organ- 
isms to estimate danger under uncertainty, in the absence 
of immediate perceptual cues. Similar to Brunswik’s in 
many aspects, an approach that focuses on the adaptive 
aspects of heuristics emerged after the heuristics-and- 
biases program had developed into one of the most pros- 
perous research programs: the fast-and-frugal-heuristics 
program initiated by Gigerenzer et al. (1999). 


6. Fast and Frugal Heuristics 


Inspired by this Brunswikian perspective and previous 
work by Edwards (1965) and Peterson and Beach (1967) 
that had drawn a more optimistic picture of the fit 
between the human mind and the principles of logic and 
probability theory, Gigerenzer and colleagues returned 
to the notion that humans can be considered rational, 
giving an adaptive interpretation to the term. This alter- 
native research program was deliberately opposed to the 
common research strategy of the Kahneman-Tversky 
program, in which cognitive performance was tested 
against the rules of classical logic and probability theory. 
Criticism of the standards against which decision mak- 
ing was assessed can be considered the starting point of 
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the alternative program on “fast and frugal heuristics” 
(Gigerenzer, 1991, 1996; Todd & Gigerenzer, 2000). 

Rejecting those normative standards as too narrow 
and unjustifiably content blind (Gigerenzer, 1996), this 
research group took a different approach to investigat- 
ing heuristics. They viewed heuristics in the light of 
bounded rationality—a term coined by Simon (1982) for 
the view of rationality that takes into account human 
limitations and implores to relate them to the structure 
of the environments in which the decisions are made 
(Todd & Gigerenzer, 2000). Accordingly, they worked on 
the assumption that heuristics offer a reasonable way 
for humans to cope with the uncertainty of the world 
by making use of the structure of information environ- 
ments. Different heuristics, metaphorically assorted in 
the “adaptive toolbox” (Gigerenzer et al., 1999), are suit- 
able for different environments. Like a tradesperson who 
chooses the tool appropriate for the task at hand, we as 
decision makers are able to determine the heuristic suit- 
able for the information structure of the decision and 
apply it. Gigerenzer and colleagues set out to discover 
such heuristic decision strategies, which environment 
structures they matched, and to what extent people 
actually use them. Instead of looking at and for heuris- 
tics as a way to explain fallacious reasoning and biases in 
judgment and decision making, they shifted the focus to 
the search for heuristics’ benefits. 

Arguably the two most important classes of heuristics 
advanced by this alternative approach, recognition and 
Take the Best, shall be illustrated in more detail in the 
following. Both of them represent single-cue heuristics 
in a Brunswikian sense. While the former always relies on 
the same primitive cue of recognition, the latter utilizes 
a more flexible, adaptive multicue toolbox, from which 
the best-suited cue is selected for the decision at hand. 


6.1 The Recognition Heuristic 

The recognition heuristic (Goldstein & Gigerenzer, 
2002) is dependent on the decision maker’s partial igno- 
rance; indeed, it illustrates the class of ignorance-based 
decision-making strategies. Faced with a choice between 
two options, having to judge which has a higher value 
on some criterion, a decision maker applying this heu- 
ristic will simply choose the option that he or she rec- 
ognizes (i.e., has seen or heard of before) as having a 
higher value than the unrecognized option. Naturally, 
the precondition for applying this heuristic is the rec- 
ognition of merely one of the options—the partial 
ignorance mentioned above.’ This heuristic will lead to 
accurate choices in environments where recognition is 
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a valid cue, positively correlated with the criterion the 
choice relates to. This may be the case in many domains, 
especially in the heavily studied paradigm of city popu- 
lation judgments, in which participants have to choose 
the city with the larger population out of two (Pachur, 
Todd, Gigerenzer, Schooler, & Goldstein, 2011). Cities 
that are recognized by many participants can indeed be 
expected to have larger population sizes. This heuristic 
affords an adaptive tool in situations characterized by 
partial ignorance, where only one option is known to 
them and more concrete knowledge is lacking (Pachur 
et al., 2011). 

However, Oppenheimer (2003) pointed out that the 
recognition heuristic may not be as simple as initially 
portrayed. Participants did not apply the heuristic blindly 
in situations in which they had knowledge of the rea- 
son for the recognition, for example, if recognized towns 
were local or if they were well known due to unusual 
circumstances, as Chernoby] is for being the location of 
a nuclear disaster. Pachur et al. (2011) rejected Oppen- 
heimer’s criticism. If participants knew that the local 
towns were small, then they had what is called “crite- 
rion knowledge,” objective knowledge about one of the 
options. If that criterion knowledge was conclusive, that 
is, “enable[d] the decision maker to deduce a solution” 
(Pachur et al., 2011, p. 7), then the recognition cues 
need not be utilized. Whether that is actually the case 
does not seem clear-cut. The fact that participants did 
not apply the heuristic when they recognized a city due 
to unusual circumstances connected to it was acknowl- 
edged and led to an extension of the recognition heu- 
ristic by another process—namely, judging whether the 
recognition cue is valid in this environment or not. This 
process comes after the recognition process and can 
explain the differential use in different environments, 
which would speak to the adaptive use of the heuristic. 

Precisely this aspect of cue validity was also tested 
directly by examining the use of the recognition heu- 
ristic across different domains or environments with 
varying recognition validity (ie., varying correlation 
between recognition and criterion). The higher the rec- 
ognition validity, the more people relied on the recogni- 
tion heuristic. Some studies even attest to a fascinating 
less-is-more effect with regard to knowledge. Those who 
know less, meaning that they recognize only a subset of 
options, may perform better under certain circumstances 
than those who recognize all options. This is because the 
latter cannot make use of the recognition heuristic but 
may not know enough about the options, beyond recog- 
nizing them, to compensate for it (Pachur et al., 2011). 
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As a cue, a special status is ascribed to recognition, 
because it is assumed to regulate the retrieval of all other 
cues. If an option is not recognized, one is unable to 
retrieve any other information about it. It can also be 
retrieved faster than any other cue. If the one cue that 
a decision is made upon is something other than recog- 
nition, one is in the realm of single-cue heuristics, the 
second class described here. 


6.2 The Take the Best Heuristic 

The second class of single-cue heuristics, also referred to 
as one-reason decision making, encompasses all heuris- 
tics that use only one cue, or reason, aside from recogni- 
tion. The decision maker compares the options’ values 
on that cue and chooses the option (among two or more 
options) with the highest value. These heuristics have in 
common the assumption that decision makers are capa- 
ble of estimating the values of the cues they are com- 
paring. Most assume that cue validity, the proportion 
of instances in which the cue correctly discriminated 
between options, can be estimated. Which of the cues 
is selected for the decision depends on the specific heu- 
ristic. Cue validity is the crucial dimension along which 
single-cue heuristics vary. 

An entire class of so-called lexicographic strategies 
relies on the assumption that cues can be sorted by 
validity, from best to worst (Luce, 1956; Payne, Bettman, 
& Johnson, 1988; Tversky, 1969). Gigerenzer and Gold- 
stein (1996) aptly termed the most prominent heuristic 
strategy from this class “Take the Best” (TTB). In TTB, 
the decision is based on the first cue in the ordering that 
discriminates between options, with the higher value on 
that cue determining the option to be chosen. 

Gigerenzer and Goldstein (1996) also specified alter- 
native ways of selecting the cue for the decision basis, 
the first of which is termed “minimalist algorithm” 
(p. 661) and selects cues at random until one discrimi- 
nates, and the second of which is called “Take the Last 
algorithm” (p. 660) and selects the cue that was used 
for the decision last time. They compare some of those 
heuristics with complex models, considered normative 
models of decision making, like the weighted-additive 
decision strategy or Dawes’ rule. Unsurprisingly, the 
more complex models could be fitted better to the data 
sets. However, more important, when the decision strat- 
egies were tested for how well they generalized to a new 
data set, TTB outperformed all other strategies in almost 
all cases. The reason for this is that their simplicity (or 
frugality) makes heuristics robust and avoids overfitting 
to one specific data set or environment. 
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7. Coherence and Correspondence 


The issue of how the effectiveness of heuristics should 
be evaluated is broader than the contrast between fit- 
ting decision strategies to given data and generalizing 
to new data sets. Laws of probability and logic are tra- 
ditionally the standard against which human decision 
making is evaluated. Rational, normative decision strate- 
gies are all coherent in that they do not violate any of 
those laws. However, heuristics do not have to adhere 
to normative rules to be able to lead to good decisions, 
illustrating the disconnect between the common evalua- 
tive standard and real-world requirements. A core aspect 
of the ABC Research Group’s work is a standard for the 
evaluation of decision strategies that is in line with the 
view of bounded rationality. Instead of applying coher- 
ence criteria that examine whether any formal norms 
were violated (as was commonplace in research inspired 
by Kahneman and Tversky), bounded rationality requires 
that heuristics, and decision strategies in general, be eval- 
uated by how well they correspond to the structure of the 
information environment in which they are used. They 
cannot be evaluated fully without at least some connec- 
tion to real-world decision situations, because it is the 
heuristic’s fit with the way the information is structured 
in the environment that needs to be assessed. The criteria 
used for this way of evaluating are called correspondence 
criteria, in contrast with the standard coherence criteria 
(Dunwoody, 2009), and include accuracy, frugality, and 
speed. 

This leads to so-called ecological rationality (Todd, 
2000), to be differentiated from “standard” rationality. 
Todd and Gigerenzer (2000) argue that this standard for 
evaluating heuristics is more sensible, because heuristics 
are not meant to be internally consistent but rather to 
provide reasonable inferences and adaptive guidance 
when time and resources are scarce. In addition, coher- 
ence criteria seem unsuitable because violations have 
not been found to be particularly costly (Arkes, Giger- 
enzer, & Hertwig, 2016). Once again, this highlights the 
immense advantage and absolute necessity of heuris- 
tic strategies: for many situations in the real world, no 
implementable standard normative strategy (optimiza- 
tion) exists, leaving heuristics as the only strategy. 


8. Critical Appraisal of Scientific Insights Gained 
from Heuristics-and-Biases Approaches 


There can be no doubt that the research program on 
heuristics and biases belongs to the most influential 
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contributions to behavioral sciences. Current psychology 
would not be what it is today without the fascinating 
work reviewed so far in this chapter. The prototypical 
concept of the human mind in basic cognitive psychol- 
ogy was largely shaped by the twofold message that (a) 
human rationality is restricted, with experts and laypeo- 
ple alike falling prey to a long list of biases and short- 
comings (Nisbett & Ross, 1980), but that (b) apparent 
violations of manifold reasoning norms can neverthe- 
less be adaptive in a complex and indeterminate world 
(Todd & Gigerenzer, 2003). Many domains of applied 
behavioral (and related) sciences have been influenced 
considerably by this research, including legal psychology 
(Gigerenzer & Engel, 2006), marketing and consumer 
research (Scholten, 1996), organizational psychology 
(West, Christodoulides, & Bonhomme, 2018), learning 
and education (Koriat & Levy-Sadot, 2001), and several 
major academic disciplines, such as economics, social 
and political sciences, and philosophy. 

However, despite these signs of success, scientific fer- 
tility, and extraordinary impact, there is also room for 
critical appraisal. Although the empirical search for biases 
and shortcomings in diverse areas has flourished for 
almost five decades, some of the most essential theoreti- 
cal underpinnings have been sorely neglected and never 
clarified. The debate about biases and alleged norm vio- 
lations is still devoid of a clearly spelled-out normative 
fundament. The conceptual background of dual-process 
theories, in which most pertinent research is theoreti- 
cally embedded, is weak and questionable. Most impor- 
tant, cogent empirical evidence from well-designed 
experiments concerning underlying mechanisms and 
delimiting conditions is conspicuously missing. Half a 
century after Tversky and Kahneman’s first publication 
(1971), we know very little about what is going on in the 
black box of judges and decision makers. 


8.1 Normative Benchmarks 

A research program that relies heavily on empirical dem- 
onstrations of biases and transgressions of probability 
theorems is strongly contingent on normative bench- 
marks. To classify a judgment or decision as biased, one 
needs to have a norm or correctness criterion against 
which empirical deviations can be tested. Biases need to 
be systematic and of practically relevant size. Therefore, 
the tasks used for research on heuristics and biases pro- 
vide participants with all information required to solve 
a probability calculation task or to choose a normatively 
optimal decision option (Edwards, Lindman, & Savage, 
1963). Participants in the coin-tossing example knew 
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they were observing a fair coin. They could have known 
that the probability of any series must be equal to that 
of any other series of equal length since its probability is 
always 2 to the power of the series’ length. The norma- 
tive solution of this task can be considered stable and 
easy to implement. 

When we take a closer look, however, we see that the 
coin-tossing example is essentially different from many 
everyday problems (Hahn & Warren, 2009). First, knowl- 
edge that a coin being tossed is fair, and thus the prob- 
abilities evenly distributed, may be a common feature in 
gambling, but it is a rare and highly improbable feature 
elsewhere. Assuming a trick coin landing heads with a 
probability of 0.8 and tails at 0.2, the task would be a 
little harder to solve normatively correctly. Moving away 
from gambling, we may end up wondering whether it is 
possible to calculate a normatively appropriate solution 
at all. We may never argue about how to calculate the 
odds of a fair coin-tossing series but may very well won- 
der how the true risk of, say, contracting the flu should 
be calculated. How can the best investment strategy on 
the stock market be determined? What is the norma- 
tively correct team to bet on in a basketball final? This 
demonstrates that there is a gap between experimental 
tasks involving betting and dice tossing, for which there 
exists an unequivocally correct solution, and real-life 
problems for which no normative solution exists, so 
that even experts and robot systems can only rely on 
approximations of normative solutions. 

Unfortunately, this intricate problem of an essentially 
normative approach was not explicated from the begin- 
ning, although the entire research project focuses on 
deviations from norms. Kahneman and Tversky (1972) 
postulated a fundamental gap between probability theory 
and human responses. Despite their initial comparison 
of cognitive illusions with perceptual illusions (Tversky 
& Kahneman, 1983), they considered the representative- 
ness heuristic to reflect an irrational illusion, the anchor- 
ing heuristic as a serious inability to ignore irrelevant 
numerical primes (Wilson et al., 1996), and sample-size 
neglect as a perfect example of irrational behavior. 

The choice of normative criterion not only created 
a problem for Kahneman and Tversky’s coherence- 
based approach to rationality but also represents an 
unresolved problem for the correspondence perspective 
taken by Gigerenzer et al. (1999) with its emphasis on 
the adaptive value of fast and frugal heuristics. Thus, 
when the Take the Best heuristic or the priority heuristic 
lead to 70% correct responses in pairwise choice tasks, 
the ultimate question is whether this can be interpreted 
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as successful in terms of corresponding to the informa- 
tion environment or as a disappointing result. What is 
the normative benchmark for rationality, or bounded 
rationality, based on correspondence criteria: 65%, 70%, 
or maybe 80% correct predictions? 


8.2 Dual-Process Theories 

As a surrogate for the missing normative or psychophysi- 
cal framework, many researchers in social cognition, but 
also in cognitive psychology, have located their work in 
the context of dual-process or dual-systems approaches 
(Chaiken & Trope, 1999; Sloman, 1996). These approaches 
postulate two systems, one that prefers fast and effortless 
thinking of the heuristic type (System 1) and one that 
can live up to systematic reasoning in line with rational 
procedures (System 2). Such a dual-process framework 
offers a seemingly plausible (but circular) explanation 
for virtually every empirical test of hypothetical heuris- 
tics. If participants exhibit the typical biases predicted 
by heuristic theories, they were apparently operating 
in System 1, presumably because time and effort were 
too costly compared to the reward for rational respond- 
ing. But if they happen to provide normatively correct 
responses, then they must have entered System 2, pre- 
sumably because they are personally involved or moti- 
vated to be accurate. Note that for such a conception to 
allow strict theory testing, a superordinate theory has 
to make clear-cut predictions about specific conditions 
under which either System 1 or System 2 is activated. 
For logical reasons alone, such a metatheory is almost 
impossible (Keren & Schul, 2009), because it is hard to 
see why System 1 and System 2 should not overlap, why 
there should be no more than two systems, and why a 
single system alone should not be able to account for 
all the evidence (Kruglanski & Thompson, 1999). In 
the absence of a metatheory that imposes strong con- 
straints on the conditions that trigger different systems, 
it remains incomplete and of restricted scientific value. 
A promising suggestion for such a metatheory is delin- 
eated by Marewski and Schooler (2011). 


8.3 Numeracy and Expertise 

Especially the initial research on heuristics in the early 
1970s did not take the perspective of dual-process theo- 
ries’ System 1: fast and loose guessing in the absence of 
motivation as well as mental and temporal resources. 
Numeracy and education level of participants have 
always been assumed to be sufficiently high. One of the 
first publications on the heuristics-and-biases project 
recruited members of illustrious psychological societies 
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as participants (ITversky & Kahneman, 1971). Further 
investigations recruited students in university courses, 
who were presumably well educated in stochastics. 
Nevertheless, cognitive biases were shown to be ubiqui- 
tous. Even experts fall prey to overconfidence (Klayman, 
Soll, Gonzalez-Vallejo, & Barlas, 1999), exhibit marked 
anchoring biases (Englich, Mussweiler, & Strack, 2006), 
and fail to engage in proper logical reasoning (Wason, 
1968). Risk assessment in highly consequential areas is 
replete with base-rate fallacies and failures to run appro- 
priate cost-benefit analyses (Swets, Dawes, & Mona- 
han, 2000). Even when it comes to taking advice from 
medical experts on existential health-related issues, 
doctors and patients do not live up to normative princi- 
ples of risk assessment and rational action (Gigerenzer, 
Gaissmaier, Kurz-Milcke, Schwartz, & Woloshin, 2007; 
Wegwarth, Gaissmaier, & Gigerenzer, 2011). 


8.4 The Conspicuous Lack of Insights on Underlying 
Cognitive Mechanisms 
Perhaps the most striking critique of half a century of 
flourishing research on cognitive heuristics, however, 
refers to the paucity of solid experimental evidence on 
underlying mechanisms. Although Tversky and Kahne- 
man (1974) introduced all of their heuristics as cognitive- 
process theories, very few experiments were conducted to 
isolate selective memory retrieval as the site of the avail- 
ability heuristic, to specify the precise similarity function 
of the representativeness heuristic (Nilsson, Juslin, & 
Olsson, 2008), or to unravel distinct constraints on the 
(insufficient) adjustment process supposed to explain 
anchoring effects (Epley & Gilovich, 2001, 2006). 
Although Gigerenzer and colleagues themselves started 
from this critique of missing experimental tests of Kah- 
neman and Tversky’s ideas, their own research suffered 
from the same paucity of unequivocal findings about 
cognitive mechanisms. There is hardly any cogent evi- 
dence on a miraculous algorithm that allows people to 
rank-order the validity of a longer list of candidate cues, 
as a precondition for Take the Best. Strict experimental 
tests of the recognition heuristic are rarely based on the 
deliberate manipulation of recognizable versus non- 
recognizable stimuli (e.g., in a paired comparison with 
selective repeated exposure). While Goldstein and Gige- 
renzer’s (2002) famous demonstration that Germans out- 
perform Americans when judging the population size of 
pairs of American cities was counted as evidence in favor 
of the recognition heuristic (because Germans know 
little about American cities except that they recognize 
some and do not recognize others), other research did 
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not support this superiority of partial knowledge (Pachur 
& Biele, 2007; Pohl, 2006). 

The cognitive algorithm supposed to underlie the pri- 
ority heuristic (Brandstatter, Gigerenzer, & Hertwig, 2006) 
is specified in all detail: when choosing between two lot- 
teries (each described by distinct outcomes o obtained 
with distinct probabilities p), the heuristic first tries to 
make a choice in favor of the lottery with the higher min- 
imal o. Only when the lotteries’ difference in minimal o 
is not sufficient, the heuristic tries to make a choice in 
favor of the lottery with the lower p(minimal 0) of a mini- 
mal outcome. Only if the difference in p(minimal 0) is 
not sufficient, the lexicographic algorithm switches to the 
highest outcome as a criterion, and only if this third cue 
still does not enable a clear-cut choice, the mechanism 
resorts to guessing. Although this is a precise algorithm, 
there is little experimental evidence available to support a 
cognitive process that follows exactly this sequence. The 
evidence marshalled in favor of the priority heuristic is, 
again, largely confined to reporting the percentage with 
which the heuristic chooses the lottery with the higher 
expected value. However, an accuracy rate of, say, 70% 
hardly reveals much about an underlying mechanism. 
When a deliberate attempt is made to test the heuristic 
experimentally, the evidence is incompatible with the 
heuristic: p(minimal 0) exerts a strong impact regardless 
of whether the preceding minimal-o cue is high or low, 
and maximal o exerts a substantial influence regardless of 
whether both preceding cues, minimal o and p(minimal 
o), are high or low (Fiedler, 2010). 

A continued debate revolves around the “adaptive 
toolbox” and its contents of many different heuristics. 
Alternative single-process models have been proposed 
as more parsimonious accounts and better fit some 
parameters, although it is very hard to test different 
approaches against each other (Glockner, Hilbig, & Jekel, 
2014; Sdllner & Bréder, 2015). The lack of explanation 
regarding the way the decision maker chooses heuristics 
from the diversely filled toolbox in different environ- 
ments is another important criticism (Cooper, 2000). Fee- 
ney (2000) points out that the choice among heuristics 
may lead to an infinite regress just like the one Gigerenzer 
and colleagues criticized about the Kahneman-Tversky 
research program. Todd and Gigerenzer (2000) admit 
that a metatheory that explains the choice of heuristics 
is sorely needed as a solution to this problem. 


9. Concluding Remarks 
By all standards, Kahneman and Tversky have certainly 


sparked a scientific revolution, and Gigerenzer and col- 
leagues have arguably achieved their goal of initiating 
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a slightly different revolution of the view of the mind. 
Other leading labs have also made strong contributions 
to research on rationality, including Fischhoff’s (1975) 
work on the hindsight bias, Einhorn and Hogarth’s (1981, 
1986) seminal papers on friendly versus wicked decision 
environments, or recent work by Hertwig and colleagues 
(Hertwig, Herzog, Schooler, & Reimer, 2008; Hertwig, 
Hoffrage, & ABC Research Group, 2013; Hoffrage & Hert- 
wig, 2012; see also chapter 8.5 by Hertwig & Kozyreva, 
this handbook) on the dialectic interplay of organisms 
and their environments. These pioneers have inspired a 
flood of research conducted in the spirit of their ideas. 
However, despite all this fruitful research and all the 
fascination it has elicited, the research program is far 
from being complete or exhaustive. We are only begin- 
ning to understand the interaction of environmental 
and cognitive sources of judgment biases and have only 
recently begun to replace the old-fashioned criterion of 
accuracy (relative to an elusive norm) by a new crite- 
rion of adaptive behavior, which is still conceptually 
underdeveloped. 

To achieve this major developmental goal, it may be 
necessary, ironically, to rid ourselves of the old paradig- 
matic idea of a normatively correct solution for judg- 
ment and decision problems. The existence of such a 
normative criterion may turn out to be more the excep- 
tion than the rule—quite like the dice-tossing problems 
and betting urns used in many traditional experiments 
are. As a consequence, there is no alternative to applying 
useful heuristics even when the smartest are motivated 
to find optimal solutions with the help of proper calcu- 
lus (Bayes’ theorem) and artificial intelligence (machine 
learning). Even under such auspicious conditions, there 
is hardly any normatively correct, conflict-free solution; 
there is always a need to find a heuristic solution that 
optimizes the adaptive value of intelligent action. 
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Notes 


1. The Ebbinghaus illusion produces a contrast effect such that 
a circle surrounded by four other circles appears larger (smaller) 
when the surrounding circles are smaller (larger). 


2. Given that Kahneman and Tversky (1974) conceived cogni- 
tive illusions as analogous to perceptual illusions, this unequal 
treatment of the two types of illusions appears conspicuous. 
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3. Along the divide of compensatory and noncompensatory 
strategies, it is located on the noncompensatory side, meaning 
that it is not traded off against additional cues. 
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2.5 Dual-Process Theories of Deductive Reasoning 
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Summary 


In dual-process models of thinking and reasoning, two 
kinds of processes are distinguished. Defining and cor- 
related characteristics of these processes as well as their 
different functions are discussed along with the tasks 
and phenomena typically addressed by dual-process 
models. The relationships between the types of processes 
and normative, instrumental, and epistemic rationality 
are considered. A selective review of specific dual-process 
models illustrates the range of dual-process models that 
have been proposed. Evidence for and against dual- 
process models is summarized and evaluated. 


1. Two Types of Thinking and Reasoning 


Dual-process models of thinking and reasoning postu- 
late that two kinds of processes are involved in reason- 
ing. These two kinds of processes differ qualitatively and 
interact or compete in generating responses to reasoning 
problems. Dual-process models of thinking and reason- 
ing are part of a larger family of such models in areas as 
diverse as social cognition, learning, and moral decision 
making. The guiding idea is that one kind of thinking, 
Type 1, delivers intuitive responses, gut feelings, or fast 
heuristic response proposals, whereas the other, Type 2, 
is responsible for slow but careful and systematic deliber- 
ations. Type 1 processes have been described as heuristic, 
context dependent, fast, and automatic; Type 2 processes 
as analytic, context independent, slow, and controlled. 
In processing a reasoning problem, Type 1 processes 
are responsible for encoding the premises, for directing 
attention to relevant aspects in a bottom-up stimulus- 
driven manner, and for proposing an intuitive response. 
Furthermore, Type 1 processes have been argued to 
replace and add premises to reasoning problems through 
pragmatic implicatures and background knowledge. 
Finally, a Type 1 response can also function as a fallback 
response if Type 2 processes fail to propose a response. 


Type 2 processes are involved in processing the task 
instructions. They assemble a task-set as an ad hoc con- 
stellation of processes coordinated to satisfy the task 
demands. Furthermore, Type 2 processes constantly 
monitor performance according to the implemented 
task-set and higher-level goals. They are responsible for 
strategic allocation of attentional resources and for pro- 
cessing that requires working-memory resources and 
mental simulation. They are activated when, for exam- 
ple, Type 1 processes do not suggest a response, suggest 
conflicting responses, or register surprise or a threat (e.g., 
Kahneman, 2011). 

Within this framework, several dual-process models 
have been proposed that differ, among other issues, on 
how precisely Type 1 and Type 2 processes are charac- 
terized. Dual-process models have to explain as a mini- 
mum what distinguishes the two kinds of reasoning and 
how they interact in generating responses to reasoning 
problems. 

Much debate has focused on which aspects distin- 
guish Type 1 from Type 2 processes; table 2.5.1 provides 
a less than complete list of distinctions compiled in 
large part from a similar table in Evans and Stanovich 
(2013a). Furthermore, there is debate on how the two 
types of processes interact. One kind of interaction is 
that Type 1 processes shape the mental representations 
on which Type 2 processes operate (e.g., Evans, 2009; 
Thompson, 2013) by drawing attention to certain fea- 
tures of the problem at the expense of others and by 
adding premises from background knowledge. Type 1 
processes also often generate a response proposal for the 
reasoning problem. In a default-interventionist model, 
this proposal serves as a default response that is acted 
upon unless Type 2 processes intervene (Evans, 2007). In 
contrast, in the parallel-competitive model, both kinds 
of processes operate in parallel, producing response pro- 
posals that compete for control of the eventual response 
(Evans, 2007). 
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Table 2.5.1 
Some features and dimensions with respect to which Type 1 
and Type 2 processes have been argued to differ 


Type 1 process Type 2 process 


Fast Slow 
Automatic Controlled 
Nonconscious Conscious 
Unintentional Intentional 
Associative Reflective 
Parallel Serial 
Heuristic Analytic 
Experiential Rational 


Does not require working 
memory 


Requires working memory 


2. Stanovich’s Tripartite Model 


It is helpful to add a number of conceptual distinctions 
to this framework. In doing so, I will draw heavily on 
Stanovich’s (2009) treatment. Stanovich (2009) defines 
the entirety of Type 1 processes as the set of autono- 
mous systems in the brain that operate autonomously 
in response to specific triggering stimuli. These comprise 
many different and heterogeneous kinds of processes 
such as 


* preattentive processes that supply content for focal 
attention and thereby shape the initial mental repre- 
sentation of the problem (Evans, 2009), 


* domain-specific processing modules as discussed by 
evolutionary psychologists (e.g., Cosmides, 1989), 

* domain-general processes of implicit learning and 
conditioning, and 

* inference, decision-making, and classification rules 
(Mercier & Sperber, 2009) that are innate or have 
been practiced to automaticity (Anderson, 1983; 
Logan, 1985). 


Type 2 processes, on the other hand, fall into two 
classes, termed the “algorithmic” and the “reflective” 
mind. The reflective level comprises control states that 
regulate processing according to higher-level goals. Pro- 
cesses at the reflective level initiate and guide processes 
at the algorithmic level. The algorithmic level imple- 
ments processes and strategies such as those required 
in solving the tasks of a test of fluid intelligence, and 
it sustains hypothetical thought and mental simula- 
tions. The algorithmic processes can also inhibit and 
override the output of Type 1 processes. The reflective 
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processes, however, initiate such overriding as well as 
mental simulations. 

Stanovich (2009) distinguishes between the algo- 
rithmic and the reflective level in part on theoretical 
grounds: they serve different functions in reasoning. 
But the distinction is also empirically motivated by 
individual-differences research on reasoning perfor- 
mance in different tasks (see Stanovich, 2009). 

Finally, Stanovich’s (2009) framework allows him to 
distinguish between different kinds of reasoning biases: 


* First of all, Type 2 processes may be not engaged. This 
can come about where a biased Type 1 output is so 
compelling (for example, associated with high feel- 
ings of rightness; Thompson, 2009) that Type 2 pro- 
cesses are not invoked or where Type 2 processes are 
not available due to lack of capacity or effort. 


* Type 2 processes may be engaged but fail to override a 
biased Type 1 response. Lack of effort is not the only 
possible cause of this outcome. Alternatively, the nec- 
essary Type 2 rules and algorithms may not have been 
learned and may therefore not be available, a situa- 
tion that Stanovich (2009) called “mindware gaps.” 


* Relatedly, the available “mindware” may be faulty 
or itself biased, as in Stanovich’s (2009) category of 
“biases due to contaminated mindware.” 


* Type 2 processes may operate on a mental representa- 
tion that does not permit one to find the correct solu- 
tion. For example, in Wason’s (1966) selection task, 
shown in figure 2.5.1, preattentive Type 1 processes 
may draw attention to the cards whose visible sides 
show the elements explicitly mentioned in the rule 
(“A” and “3” in figure 2.5.1), which do not include 
one of the cards that needs to be selected (“1” in 
figure 2.5.1). Any reasoning focusing on only these 
cards would probably be seen as an instance of ratio- 
nalization of an initial response (Evans, 1996; Evans 
& Wason, 1976). Stanovich (2009) terms this kind of 
Type 2 reasoning “serial associative cognition with a 
focal bias.” 


3. Type of Process and Rationality 


3.1. Instrumental, Normative, 

and Epistemic Rationality 

What is the relationship of the different processes to 
human rationality? Consider instrumental rationality, 
defined as choosing suitable means for one’s ends. Type 
1 processes, if defined as autonomous and triggered by 
specific cues, have no clear relationship to instrumental 
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The (reduced) base-rate task 
(after Pennycook, Fugelsang, & Koehler, 2015) 


General Instructions: 

In a big research project a large number of studies 

were carried out where short personality descriptions 

of the participants were made. In every study there were 
participants from two population groups (e.g., carpenters 
and policemen). In each study one participant was 
drawn at random from the sample. You’ll get to see a 
personality trait for this randomly chosen participant. 
You'll also get information about the composition of 

the population groups tested in the study in question. 
You'll be asked to indicate to which population group the 
participant most likely belongs. 


One Trial: 

This study contains: politicians and nannys. 

Person ‘A’ is dishonest. 

There are 999 nannys/5 politicians. 

Is person ‘A’ more likely to be a) a politician or b) a nanny? 


XX vA 
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The belief-bias task 
(after Handley, Newstead, & Trippas, 2011) 


General Instructions: 

You must assume each premise is true (even if in 
reality it is not true) and respond with the answer 
which logically follows from the statements presented. 


For example: If you finish a drink, then the drink will be full. 


Suppose that you finish your drink. 
Will your drink be empty? 


The logic-based answer to this problem is NO, because 
you must assume that if a drink is finished, then it will 
be full; therefore, the glass will not be empty. 


One Trial: 

Ifa child is crying, then he is happy. 
Suppose a child is crying. 

Is the child happy? 


/ 


The Wason selection task 
(after Klauer, Stahl, & Erdfelder, 2007) 


Instructions: 


Below you see a number of cards from a set of cards. Each card in the set has a capital letter on one side and a digit on the other. Naturally, 
only one side is visible in each case. For the set of cards, a rule has been stated. It is: “If there is an A on the letter side of the card, then there 
is a3 on the number side.” You must decide which card(s) displayed would have to be turned over in order to test the truth or falsity of the rule. 


A B 


%\ 


Figure 2.5.1 


Three tasks frequently used in research on dual-process models. 


rationality as they are not directly informed by one’s 
goals. The same is true for normative rationality, defined 
as the adherence to given norms. There are, however, a 
couple of considerations that qualify these strong state- 
ments. Type 1 processes, it is now widely held, can also 
deliver normatively correct responses if they are well 
attuned to environmental regularities, and they can 
be sensitive to abstract structural features of a task. For 
example, the modus ponens rule describes the inference 
from a major premise of the form “If p then q” and a 
minor premise “p” to the conclusion “q.” Modus ponens 
may be so well practiced that it can often be triggered 
habitually and fast in reading and comprehending the 
premises (e.g., Lea, 1995; Lea, O’Brien, Fisch, Noveck, & 
Braine, 1990; Rader & Sloutsky, 2002). 

Relatedly, Type 1 processes are frequently argued to 
allow us to approach major goals that we hold (e.g., 
Evans, 2014), linking them to instrumental rationality. 
This is sometimes associated with dual-systems views 


or two-minds theories in which Type 1 and Type 2 pro- 
cesses live in different systems, linked to different evo- 
lutionary stages (e.g., Evans, 2014; Mercier & Sperber, 
2017; Stanovich, 2004). 

For example, integrating one’s beliefs about the likeli- 
hood of possible outcomes of acts and one’s evaluations 
of these outcomes is seen as a hallmark of instrumen- 
tal rationality in decision theory (Buchak, 2013). At the 
same time, relying on one’s beliefs underlies the genera- 
tion of belief bias (Wilkins, 1928) in deductive reason- 
ing, which has sometimes been construed as reflecting 
the operation of a Type 1 process. Similarly, Gigerenzer 
and Todd (1999) have argued (a) that human beings 
have at their disposal a toolbox of fast and frugal heu- 
ristics, some of which would probably qualify as Type 
1 processes for some authors, and (b) that these often 
mimic optimal strategies for achieving current goals 
by capitalizing on regularities in benign environments. 
And so, instrumental rationality alone is unlikely to be 
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a good feature for cleanly distinguishing Type 1 from 
Type 2 processes. Similar points can be made regarding 
normative rationality. 

Epistemic rationality regards the internal coherence 
and consistency of the beliefs that we hold and the sup- 
port that we seek for them in terms of external evidence. 
In his two-minds theory, Evans (2014) argues that epis- 
temic rationality is something pursued only by what he 
calls the “new” mind and Type 2 processes operating on 
explicit knowledge. In contrast, both the “old” mind, 
relying on Type 1 processes, and the “new” mind work 
in concert to achieve instrumental rationality. The next 
section explores the idea that justifications as an ele- 
ment of epistemic rationality might reliably distinguish 
between Type 1 and Type 2 processes. 


3.2 Justification as a Qualitative Distinction 

between Type 1 and Type 2 Processes? 

Consider an analogy to Plato’s definition of knowledge 
as justified true belief. Type 1 processes can deliver cor- 
rect responses (true belief), satisfying the requirements 
of instrumental rationality and sometimes normative 
rationality. But they do so for the wrong reasons (i.e., 
unjustified) inasmuch as they capitalize on environmen- 
tal features that happen to covary with the normative or 
rational response most of the time. This can be revealed, 
for example, in experimental situations in which the 
auspicious environmental features are intentionally 
removed and in which responses then depart from the 
normative prescriptions or from those of instrumental 
rationality. Kahneman and Tversky’s heuristics-and- 
biases program (see chapter 2.4 by Fiedler, Prager, & 
McCaughey, this handbook) has been characterized as 
having achieved just this (Kahneman, 2011). For exam- 
ple, one’s beliefs will often provide an adequate guideline 
when judging the validity of the conclusion of a logical 
argument, but they may be misleading when the task is 
to disregard one’s background knowledge and to judge 
the validity solely based on whether the proposition fol- 
lows logically from the premises of the argument, taking 
these as true (see figure 2.5.1 for an example). 

And thus, pursuing the above analogy a bit further, 
it may not be enough to focus on the correctness or rea- 
sonableness of the outcomes of the reasoning process. 
Instead, the outcome should also come about for the right 
reasons. One way to ensure this is to require that reasoners 
should be aware of reasons or justifications for the particu- 
lar reasoning steps they engage in. This requirement to be 
able to justify one’s responses as they are generated (rather 
than post hoc) captures nicely the distinction between 
Type 1 processes, characterized as intuitions being based 
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on gut feelings or reactions, and deliberate Type 2 pro- 
cesses, based on reflection. In a similar vein, Mercier and 
Sperber (2009) suggest distinguishing between intuitive 
and reflective inferences. Reflective inferences involve 
the representation of reasons for the inferred conclusions 
(see also Mercier & Sperber, 2017). Of course, conclusions 
that come with justifications need not be normatively 
or instrumentally correct, for reasons already discussed 
above, but this does not diminish their potential as a 
demarcation criterion. 


4. Examples of Dual-Process Models and Related 
Models in the Reasoning Literature 


4.1 Evans and Stanovich’s Framework 

Let us consider a number of dual-process models in the 
reasoning domain to illustrate these concepts further. 
Evans and Stanovich (2013a) have provided a frame- 
work for dual-process models in reasoning and decision 
making in which the two types of reasoning are distin- 
guished in terms of two defining features: Type 2 pro- 
cesses load working memory, whereas Type 1 processes 
do not require working-memory resources. In addition, 
another hallmark of Type 2 processing is that it sustains 
hypothetical thought or cognitive decoupling (Stano- 
vich, 2009), “the ability to distinguish supposition from 
belief and to aid rational choices by running thought 
experiments” (Evans & Stanovich, 2013a, p. 236). In par- 
ticular, working memory and cognitive decoupling are 
involved when Type 1 processes and their outputs are to 
be inhibited. 

The interaction between the two types of processing 
is described in terms of a default-interventionist model: 
Type 1 processes deliver a default intuition that can be 
overridden by Type 2 processes. Whether and when 
this occurs is a function of motivation, the availabil- 
ity of working-memory resources, and individual dif- 
ferences in thinking dispositions (Evans & Stanovich, 
2013b). Type 2 processing can be algorithmic, propos- 
ing an alternative response to the default intuitive one 
or affirming it, and reflective, embodying goal-driven 
higher-level control responsible for the decision to over- 
ride Type 1 processing. Evans and Stanovich (2013b) see 
this framework as a paradigm or metatheory that can 
inspire task-level theories. 


4.2. A Single-Function Dual-Process Model 

Oaksford and Chater (2010, 2012) propose what they 
called a “single-function dual-process model” targeted 
at reasoning with conditionals. Type 1 processes in this 
model are the processes of spreading activation in a 
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constraint-satisfaction network identified with long-term 
memory. Type 2 processes operate on both long-term 
memory and working memory and are responsible for 
strategic queries of long-term memory, for representing 
the outputs of these queries in working memory, and for 
operations on these working-memory contents. 

Upon encountering a conditional sentence of the 
form “If p then q,” a part of long-term memory is acti- 
vated, including the nodes for the antecedent (p) and the 
consequent (q) as well as other representations strongly 
linked with them. Given a minor premise, such as the 
antecedent in a modus ponens inference, reasoners will 
clamp on the node representing the minor premise and 
monitor the flow of activation in the activated portion 
of long-term memory until an equilibrium is reached. 
The activation of the node representing the consequent 
is then read off and the result of the Ramsey-style test 
(Bennett, 2003) stored in working memory. This repre- 
sentation in working memory is actually conceived of as 
similar to the mental-model representations in Johnson- 
Laird’s mental model theory (see chapter 2.3 by Johnson- 
Laird, this handbook). 

Strategic queries of long-term memory allow the 
constraint-satisfaction network to compute activation 
levels for hypothetical and counterfactual premises. Type 
2 processes are also responsible for undoing changes in 
long-term memory due to a previous query, and they can 
have a decoupling function in screening the representa- 
tions in working memory from the intrusion of contextu- 
alized long-term memory information. This can result in 
processing that mimics logical thought as a limiting case 
of the function primarily computed, namely conditional 
probabilities. Nevertheless, the two kinds of processes 
serve a single function: to assess probabilities. 

Type 1 and Type 2 processes thus differ on several 
of the features listed in table 2.5.1, most conspicuously 
in terms of whether working memory is required. Fur- 
thermore, although Oaksford and Chater (2010) do not 
explicitly say so, processes of spreading activation in 
long-term memory would presumably be nonconscious 
and, once triggered by stimuli or a Type 2 query, run to 
completion automatically. In contrast, Type 2 processes 
would probably be seen as intentional and conscious 
attempts to probe long-term memory and to represent 
and integrate mental-model-like representations in 
working memory. The interaction of the two processes 
is twofold: Type 1 processes determine which parts of 
long-term memory are activated, shaping the problem 
representation. Furthermore, the interaction can take 
the default-interventionist form in that Oaksford and 
Chater (2010) assume that a default query of long-term 
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memory can occur stimulus driven and provide a default 
assessment of the relevant probabilities that can how- 
ever be modified by Type 2 queries of long-term memory 
to the point where the default assessment is completely 
inhibited. 


4.3 A Dual-Strategy Model 

Verschueren, Schaeken, and d’Ydewalle (2005a, 2005b) 
have proposed a dual-strategy model of conditional 
reasoning that was further developed by Markovits and 
colleagues (e.g., Markovits, Brunet, Thompson, & Bris- 
son, 2013; see also chapter 15.1 by Markovits, this hand- 
book). The idea is that people can use a combination of 
statistical and counterexample forms of reasoning. Mar- 
kovits et al. (2013) align the statistical strategy with the 
assessment of probabilities similar to the stimulus-driven 
long-term memory queries considered by Oaksford and 
Chater (2010, 2012). The statistical strategy is to accept 
conclusions that are assessed as probable (somewhat like 
in accepting believable conclusions in belief-bias prob- 
lems, discussed in more detail below). It is associated 
with little cognitive cost, operates fast, and delivers a 
default response. Given time and cognitive capacity, the 
statistical strategy can be replaced by a counterexample 
strategy in which a conclusion is rejected if a counterex- 
ample can be found. Counterexamples can be implied by 
the given information, but as in Oaksford and Chater’s 
(2010, 2012) single-function dual-process model, coun- 
terexamples can also be generated via a strategic search 
of relevant background knowledge. The counterexample 
strategy is slower, and it comes with a higher cognitive 
cost. One difference to Oaksford and Chater’s approach 
is that the two kinds of processes compute different 
functions, the former being sensitive to probabilistic 
information, the latter being driven by the existence or 
absence of at least one counterexample. The approach is 
thus a dual-function dual-process one. 


4.4 A Dual-Source Model 

Consider finally the dual-source model of probabilis- 
tic conditional reasoning by Klauer, Beller, and Hitter 
(2010; see also Singmann, Klauer, & Beller, 2016). The 
model distinguishes between two processes in terms of 
whether they derive response proposals based on back- 
ground knowledge or based on the form of the argu- 
ment. The model is not committed to further possible 
distinctions between the two types of processes. Instead, 
it provides a methodology and experimental paradigm to 
disentangle the contributions of both kinds of processes, 
making them and their features amenable to empirical 
study. 
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5. Evidence in Favor of and Against 
Dual-Process Models 


5.1 Working-Memory Load, Speeded Responses, 

and Patterns of Interference 

Evidence in favor of dual-process models of reasoning has 
focused on tasks such as those shown in figure 2.5.1. In 
a classical belief-bias task, participants are to judge the 
validity of valid and invalid syllogisms with believable 
and unbelievable conclusions. Belief bias is the finding 
that participants are more likely to accept as logically valid 
syllogisms with believable conclusions than syllogisms 
with unbelievable conclusions. As already mentioned, the 
assessment of believability has been argued to be based 
ona Type 1 process that is fast and requires little working- 
memory involvement, whereas logic-based responses 
require slower and working-memory-dependent Type 2 
processes. In line with these ideas, both time pressure 
(Evans & Curtis-Holmes, 2005) and working-memory 
load (De Neys, 2006) decreased participants’ ability to dis- 
criminate between valid and invalid syllogisms, whereas 
the effect of believability was increased (Evans & Curtis- 
Holmes, 2005) or not affected (De Neys, 2006). 

On the other hand, Handley, Newstead, and Trip- 
pas (2011) also asked a group of participants to judge 
the believability of the conclusion and found (a) that 
under some conditions, judgments of believability took 
more time than judgments of logical validity and (b) 
that judgments of believability were interfered with by 
the validity of the syllogism in question: conclusions 
of valid syllogisms were judged more believable than 
conclusions of invalid syllogisms. This suggests a more 
symmetric role for the belief-based and the logic-based 
process than the studies just cited. 

Let us take a closer look at the actual problems 
employed. Evans and Curtis-Holmes (2005) and De 
Neys (2006) employed problems with conclusions that 
were either believable or unbelievable. For example, a 
believable conclusion was “Some healthy people are not 
astronauts”; a (relatively more) unbelievable conclusion 
was “Some astronauts are not healthy people.” Judging 
the believability of these conclusions is probably driven 
by a spontaneous, stimulus-triggered memory query 
of the kind described by Oaksford and Chater’s (2010) 
dual-process models. In contrast, Handley et al. (2011, 
experiments 1 to 3) employed the belief-bias problems 
exemplified in figure 2.5.1 in which the conclusion 
(e.g., “The child is happy”) is neither believable or unbe- 
lievable. It only acquires a value on the believability 
dimension once the minor premise (“The child is cry- 
ing”) is integrated with relevant conditional background 
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knowledge (such as “If a child is crying, then he or she is 
not happy”). Note that assessing the believability of the 
conclusion thereby requires almost the same processing 
steps as assessing its validity, the only difference being that 
the minor premise needs to be integrated with the explic- 
itly stated conditional rule (“If a child is crying, then he is 
happy”) in checking validity, whereas it needs to be inte- 
grated with conditional background knowledge in assess- 
ing believability. In short, assessing validity and assessing 
believability appear to have been equated in terms of pro- 
cessing demands, questioning their alignment with dif- 
ferent types of processing. Note also that this issue does 
not afflict Handley et al.’s experiments 4 and 5, which are, 
however, subject to a similar alternative account in terms 
of aftereffects of switching between assessing believability 
and validity from trial to trial (Handley et al., 2011, p. 40). 
Switching is known to inflate existing, and sometimes 
newly create, crosstalk between the two alternating tasks 
(Allport, Styles, & Hsieh, 1994). 

Similar ideas have been further developed in Chun 
and Kruglanski’s (2006; see also Kruglanski & Gigerenzer, 
2011) work on the base-rate task. In base-rate problems, 
participants are to decide which of two professional cat- 
egories a person is more likely a member of. Two kinds of 
information are provided: first, participants are told that 
the person in question is randomly drawn from a sample 
of persons with skewed base rates for the two categories. 
For example, there might be 800 lawyers and 200 engi- 
neers. The second source of information is a description 
of traits and/or habits of the person such as that the per- 
son is not married, is introverted, and likes to spend his 
free time reading science fiction and writing computer 
programs. The second source is thus a description that 
is stereotypical of members of one of the two catego- 
ries, engineers in the present example. Relying on the 
stereotypical information is usually considered heuris- 
tic, processing the base-rate information can be analyti- 
cal. A typical finding is that the base-rate information is 
neglected and only the stereotypical information used 
(Kahnemann & Tversky, 1973). 

Chun and Kruglanski (2006) argue, however, that the 
decisive difference between the two kinds of information 
is not related to the different contents (base rates versus 
stereotypes) but to how easy it is to process them. In 
general, equating how easy it is to process the two kinds 
of information should make their impact on judgments 
in the base-rate task more symmetrical, as Chun and 
Kruglanski (2006) demonstrate across several studies. 

We argued that this partly explains Handley et al.’s 
(2011) results: the task demands of some of their belief- 
bias problems equated processing ease for judging validity 
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and judging believability. Indeed, when more difficult 
logical problems are used, the effects of validity on judg- 
ments of belief decrease, as shown by Trippas, Thomp- 
son, and Handley (2017), although these studies are also 
afflicted by the abovementioned switching confound 
that is known to inflate crosstalk between the two alter- 
nating tasks. 

A similar issue may be in place for many recent stud- 
ies using the base-rate task (e.g., Bago & De Neys, 2017; 
De Neys, 2012, 2014; De Neys & Glumicic, 2008; New- 
man, Gibb, & Thompson, 2017; Pennycook, Fugelsang, 
& Koehler, 2015; Pennycook & Thompson, 2012; Pen- 
nycook, Trippas, Handley, & Thompson, 2014). In the 
“reduced” version shown in figure 2.5.1, the issue is 
especially clear: base-rate information and stereotypical 
information are both presented in one sentence. Both 
can suggest responses via simple rules or heuristics—to 
pick the larger category for the base-rate information 
and to pick the matching category for the stereotypical 
information. Again, ease of processing appears to be 
closely comparable for the two heuristics, questioning 
their alignment with different kinds of processing. 

For example, Pennycook et al. (2015) erect a three- 
stage dual-process model on complex patterns of inter- 
ferences in response times and response rates in the 
reduced base-rate task. But as pointed out by Krajbich, 
Bartling, Hare, and Fehr (2015), these patterns of interfer- 
ence can also come about through simpler one-process 
models in which the strength of preference (operation- 
alized in terms of relative choice frequency) for one of 
the responses (the stereotypical response versus the base- 
rate response) and response time are related so that the 
preferred response is generally given faster the stronger 
the preference for it. Pennycook, Fugelsang, Koehler, 
and Thompson (2016) acknowledge this argument and 
point to aspects of their data pattern that may still be 
difficult to account for by this alternative account. 

A number of these studies also used a two-response 
paradigm (e.g., Bago & De Neys, 2017) in which par- 
ticipants give a first response as quickly as possible fol- 
lowed by a second response without time pressure. One 
major finding is that most participants stick to their first 
responses, so that changes from first to second response 
are rare. This has been argued to question a number of 
dual-process models: if the first response relies on Type 1 
processes, whereas Type 2 processes are involved in the 
second response, changes should regularly occur when 
both kinds of processes suggest different responses. One 
clue to what is happening here may be that participants 
appear to be very consistent in their responses to the 
problems (e.g., De Neys, 2017a, p. 56): across problems, 


179 


they either rely on base-rate (or in other cases logical) 
information or on stereotypical information, suggesting 
that participants decide and perhaps justify to them- 
selves early which of these two sources of information to 
rely on. They then stick to this deliberate decision rather 
than reevaluate it for each problem anew. If both sources 
of information are easy to process, both strategies are 
likely already available for the first response. 

In fact, when problems are used in which logic and 
believability, or base-rates and stereotypical information, 
are arguably less strongly equated in processing difficulty, 
changes are more frequent (Thompson, Prowse Turner, 
& Pennycook, 2011). In addition, in Thompson et al.’s 
(2011) studies, feeling-of-rightness (FOR) judgments 
for the first responses significantly predicted how often 
changes occur and how long it takes to produce the sec- 
ond response after the first. In a default-interventionist 
dual-process framework, this is consistent with the idea 
that FOR determines the likelihood and intensity of the 
engagement of Type 2 processes. 


5.2 Correlational Evidence 

Another source of evidence is individual-differences 
research. Stanovich and West (2000; see also chapter 15.2 
by Stanovich, Toplak, & West, this handbook) in partic- 
ular have investigated correlates of task performance in 
many tasks in which the modal response deviates from 
the one considered normative. In many of these tasks, 
measures of intelligence correlated positively with nor- 
mative correctness of the response and negatively with 
giving the modal response. Importantly, predictions 
can be made for where to expect such correlations (e.g., 
when Type 2 processing is substantially involved in a 
version of the task) and where not (e.g., when Type 1 
processing usually generates the correct response with- 
out conflict with Type 2 processing)—predictions that 
have been confirmed in several instances (Evans & Sta- 
novich, 2013a, 2013b). 


6. Criticisms of Dual-Process Models 


6.1 Distinguishing Type 1 and Type 2 Processes: 

The Dichotomous and the Gradual View 

Criticisms of dual-source models have addressed many 
aspects of dual-process models (Keren, 2013; Keren & 
Schul, 2009; Kruglanski & Gigerenzer, 2011). A number 
of problems identified by Keren and Schul (2009) and 
Keren (2013) arise from the fact that dual-process models 
have historically characterized the two types of processes 
in terms of arrays of attributes (see table 2.5.1) that are 
not perfectly correlated, an approach explicitly eschewed 
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in recent theoretical expositions of dual-process theory 
(e.g., Evans & Stanovich, 2013a). 

This reminds one of earlier debates about the defini- 
tion of automaticity (for a review, see Moors & De Hou- 
wer, 2006): some definitions understood the contrast 
between automatic and controlled processes as a dichot- 
omy (Shiffrin & Schneider, 1977) with different clusters 
of features characterizing automatic and controlled pro- 
cesses, respectively, much like exemplified in table 2.5.1 
for Type 1 and Type 2 processes. Other definitions con- 
sidered a continuum of attentional requirements with 
automatic processes at one end and nonautomatic pro- 
cesses at the other (e.g., Anderson, 1983). The dichot- 
omous view was challenged by studies finding lack of 
correlation among central features ascribed to automatic 
processes, such as that they are efficient, unintentional, 
uncontrollable, and unconscious (Bargh, 1992). Such 
challenges led to a decompositional view according to 
which different automaticity features are to be studied 
separately and independently, suggesting in the end the 
deconstruction of the concept of automaticity. 

Gradual views, on the other hand, maintain the con- 
cept and align it with a transition from algorithmic com- 
putation to single-step memory retrieval (Logan, 1988) 
or algorithmic strengthening in skill acquisition (Ander- 
son, 1983). Logan (1985) argued that the different auto- 
maticity features have different time courses of change 
with practice, and thus lack of co-occurrence among the 
features is no surprise. The gradual view would only be 
challenged if different features were found to develop 
in different directions as practice increases. Whereas the 
gradual view makes it difficult to draw a line distinguish- 
ing automatic from nonautomatic processes, another 
response was to raise one dichotomous feature to a 
definitional status and acknowledge that other features 
might, but need not, co-occur with it (Bargh, 1992). 

Similar arguments and moves can be found in debates 
on dual-process models. For example, some, although 
not all, of the features in table 2.5.1 are inherently 
gradual and define processing continua rather than 
qualitative distinctions (Keren, 2013; Keren & Schul, 
2009). Involvement of working-memory resources, a 
feature highlighted by Evans and Stanovich (2013a), 
clearly comes in degrees, and it is notoriously difficult 
to distinguish between small and zero involvement. 
In a similar vein, Kruglanski and Gigerenzer (2011; see 
also Chun & Kruglanski, 2006) acknowledge that differ- 
ent processes differ on many qualitative aspects such as 
the contents that they operate on but that graded dif- 
ferences in processing ease and in perceived ecological 
validity may be responsible for relatively greater impacts 
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on the ultimate response of some processes than others. 
Cognitive decoupling and mental simulations, another 
distinguishing feature highlighted by Evans and Stano- 
vich (2013a), as well as the requirement to provide rea- 
sons (Mercier & Sperber, 2009), may be more difficult to 
construe as gradual distinctions, depending upon how 
exactly these features are defined and operationalized, 
and they may serve as defining features if a dichotomous 
view is preferred. 


6.2 The Relative View 

Alternatively, the conceptualization of dual-process 
models could also be based on a relative view as pro- 
posed by Moors and De Houwer (2006) for the distinc- 
tion between automatic and controlled processes. This is 
a refinement of the gradual view in which a standard of 
comparison is introduced. In terms of dual-process mod- 
els, a process would be called Type 1 relative to a stan- 
dard of comparison appropriate to a given task context, 
whereas in another context, the same process might be 
classified as Type 2 relative to a standard of comparison 
that is prevalent in that context. For example, in the 
context of the Stroop task, a word-reading and a color- 
naming process are involved. When the task is to name 
the color in which a word is written, uninstructed, spon- 
taneous reading of the word interferes with naming the 
word's color if the word itself denotes a different color. In 
contrast, if the task is to read the word, there is no analo- 
gous interference from the word’s color in conflict trials 
(MacLeod, 1991). To account for this asymmetry, word 
reading has been considered relatively more automatic 
(Type 1) and color naming relatively more controlled 
(Type 2; Lindsay & Jacoby, 1994). In the tasks studied 
in the thinking-and-reasoning literature, both processes 
would, however, probably be considered Type 1 pro- 
cesses relative to the resource-depleting reasoning algo- 
rithms demanded by many reasoning tasks. Similarly, 
assessing believability in belief-bias tasks may normally 
tax cognitive resources so much less than assessing logi- 
cal validity that the relative view would designate the 
former process Type 1 and the latter Type 2. If and when 
assessing believability is, however, made more difficult, 
and assessing logical validity easier, thereby equating 
both in terms of processing demands (as in Handley 
et al., 2011, experiments 1 to 3), the gradual view would 
deny such a designation in this context. 


7. Conclusion 


Keren (2013) argues that single-process accounts may be 
offered for a number of dual-process phenomena (see 
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also Chun & Kruglanski, 2006; Krajbich et al., 2015; Krug- 
lanski, 2013). Proponents of dual-process models need 
to defend their theories against this criticism, but the 
criticism is targeted at specific instances of dual-process 
models for specific tasks. At a more abstract level, dual- 
process models have proven to provide an immensely 
fruitful framework for integrating a diversity of related 
findings. They serve as a template for devising specific 
accounts of tasks of interest, providing a rich toolbox 
of concepts and process assumptions (Evans & Stano- 
vich, 2013b). The framework has thereby generated an 
enormous amount of research producing new tasks and 
revealing new phenomena that might not have been 
uncovered without it, many of them summarized in the 
recent book by De Neys (2017b). There is little doubt 
that the core ideas and concepts of dual-process models 
will continue to be influential and to open up new and 
exciting avenues of research. 
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Summary 


Taking a broad historical line, we discuss major aspects 
of rationality that can be analyzed in a logical and com- 
putational perspective. Topics include classical notions 
from the foundations of computability, insights from 
the development of computer science and AI, and the 
richer picture of rationality emerging in current logi- 
cal studies of agency. We also discuss two challenges: 
distributed computing and evolutionary games replace 
“high rationality” by “low rationality” in behavior; 
machine learning defies classical views of representation 
and inference. Both trigger new logical themes in the 
study of rationality. 


1. Logic and Rationality 


Rational behavior is a rich phenomenon, not captured 
in a single formula, but mapped out in this entire hand- 
book. Let us take the commonsense view. We all believe, 
or try to believe, that our behavior is driven by reasons 
and reasoning and that we are susceptible to reason, 
changing our minds when confronted with new facts 
or considerations. This is how we see ourselves, how we 
justify our actions to others, and how academic organ- 
izations present themselves to a general public. Further 
important aspects of rationality, such as preferences and 
goals, will only be touched upon lightly in this chapter. 

Since Antiquity, reasoning has been at the heart of 
logic. In fact, logic is often seen as rationality in its pur- 
est, perhaps its most intimidating, form. We will chart 
this match, without claiming logic is all there is to 
rationality. 

Example: Valid and invalid consequence. Classical logic 
tells us things like this: an inference to ~B from AB 
and ~A is invalid (B might hold for other reasons than 
A)—but the inference from AB and —B to “A is valid 
and in fact the engine of refutation. Valid inferences, 
put together, form complex proofs that can yield sur- 
prising new insights. Over time, studying all this has 


produced a rich semantic and syntactic discipline of 
logical systems. 4 

Before we start, here is a distinction. Logical proof can 
be seen as a practical engine of rational behavior, and 
different logical systems can model rational reasoning 
practices. But there is also a theoretical foundational use 
of logic, as a study of the structure of rationality: its laws 
and its limitations. In this second sense, logical analysis 
can target any practice that rational agents engage in: 
not just reasoning but also observing, taking decisions, 
or debating. Both uses, practical and theoretical, will 
occur in what follows. 

But here is a further step. Logical systems for analyz- 
ing reasoning are cultural artifacts that interact with 
human practices. In particular, they feed into compu- 
tational devices. For instance, the valid inference from 
A-—B and —B to ~A is also a basic law of binary arith- 
metic, at work in one’s computer. This association has 
proved fruitful in the foundations of mathematics, and 
practically, it has triggered the development of comput- 
ers, information technology, and artificial intelligence, 
which are transforming our world. Some even fear that 
our logical tools have started overtaking us. 

A full picture of reasoning requires that “us.” Many 
themes in this chapter connect naturally to empiri- 
cal cognitive psychology. This interface is beyond our 
scope, and we refer the reader to the empirical entries 
in this handbook. Instead, we proceed to the logico- 
computational perspective on rationality. 


2. Mini-History of Reasoning and Computation 


To understand the links between logic and rationality, 
a historical perspective is helpful (cf. Kneale & Kneale, 
1962). Over time, many forms of reasoning were cap- 
tured in logical systems by philosophers, mathemati- 
cians, and others—a process that is still continuing. Once 
discovered, these systems became intellectual tools that 
enhanced rational thinking. Then, in the early modern 
age, Llull and Leibniz realized that reasoning is close to 
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computation. From there runs a straight road to the logic 
machines of Babbage and Lovelace and onward to mod- 
ern computers and artificial intelligence (AI) systems. On 
the way, the notion of computation acquired sharper con- 
tours. A computing device need not be tied to one specific 
task; it can be programmable, the way a loom can weave 
different textiles depending on its book. Thus, comput- 
ing means finding algorithms performing tasks, and since 
algorithms work on code, it also means finding data struc- 
tures that represent information in appropriate ways. 

All this is similar to the reality of logic itself. Text- 
books say that logic is the study of inference or reason- 
ing, but much more is involved. Reasoning presupposes 
a vehicle, often a language, representing the notions 
one reasons with. Thus, as noted in the perceptive essay 
(Beth 1971), logic has always been about a tandem of 
proof and definition or, if you wish, proof theory and 
model theory. Moreover, Beth added as a crucial third 
historical constant of logical thought the notion of algo- 
rithm, which combines the former two. 

The history of computing is remarkable in that major 
principles were discovered before practical success, 
unlike in many other fields. Gédel (1931/1986) analyzed 
the limits of what logical proof systems can achieve, 
finding that systems whose expressive power suffices 
for encoding basic arithmetic are either inconsistent or 
incomplete: unable to prove all intuitive mathemati- 
cal truths about their domain. Gédel’s proof involved a 
deep analysis of computable (“recursive”) functions, and 
subsequently, Turing (1936) defined machines that can 
compute all recursive functions. There is even a universal 
Turing machine that, given any program code and input, 
computes the effects of running that program on that 
input. In this setting, Church (1936) showed that stan- 
dard reasoning systems such as first-order logic, although 
axiomatizable, are undecidable: no computing method 
can decide for arbitrary first-order consequence problems 
whether they are valid. Thus, a major trade-off came into 
view: increased expressivity of a language and complex- 
ity of its decision problem for validity are at odds. 

This history highlights several points that seem crucial 
to understanding the nature and scope of rationality even 
today. The first is a practical issue of the modus operandi: 
if rationality has a computational engine, how should 
we understand its interplay of reasoning, information, 
and concept formation? The second point is theoretical: 
are there principled limitations to logical rationality— 
say, are some natural tasks beyond the scope of rational 
inquiry? Gédel’s theorems keep generating discussion 
(Wang, 1996), and a common moral is that there is more 
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to rational thinking than what is captured in logical sys- 
tems. Even so, whenever this “more” is explained sys- 
tematically, the limitation theorems apply again. Finally, 
results on what proof systems and computing devices can- 
not do were in fact immensely helpful in the further devel- 
opment of systems of inference and computation that can 
do a lot. Likewise, the modern “challenges to rationality” 
discussed in this chapter may actually yield new insights 
into what rationality can achieve. Having said that, much 
current literature in AI or cognitive psychology is of the 
“can do” type: one seldom reads about exciting discover- 
ies of deep new limitations. 

The foundational era of Gédel and Turing showed what 
is provable or computable in principle. While this high 
abstraction level remains a valid perspective on rational- 
ity, subsequent history tells us many further things. 


3. Computer Science and Artificial Intelligence 


3.1 Computer Science 

The development of computing in the 20th century has 
generated major practical achievements but also an ever- 
growing insight into fine structure. There are different 
models for computation, from Turing machines to more 
recent formats for computing, and crucially, these mod- 
els come in hierarchies. Some tasks are solved by simple 
finite automata; others require memory management to 
varying degrees. Likewise, there is a wide diversity of, 
poorer or richer, languages for specifying data structures 
and writing programs (Harel, 1987). Next, significantly, 
around 1980, computing architecture moved away from 
single Turing machines to distributed networks, the real- 
ity of computing today (Andrews, 2000). These develop- 
ments have given rise to new fields such as automata 
theory (Chakraborty, Saxena, & Katti, 2011), complexity 
theory (Papadimitriou, 1994), and process algebra (Berg- 
stra, Ponse, & Smolka, 2001), which chart the varieties of 
computation in different ways, many of them connected 
to logic. This process is still ongoing, and the founda- 
tions of computation remain under debate. For instance, 
there is no consensus on a definitive notion of algo- 
rithm, a more intensional notion than an extensional 
input-output record of Turing machines (Bonizzoni, 
Brattka, & Lowe, 2013; Haugeland, 1997). Computation 
today is rather a way of producing behavior. 

All this comes with notions and insights that are rel- 
evant to rationality. The fine structure of computation 
gives a precise sense to the earlier double-edged modus 
operandi: reasoning engine and representational appa- 
ratus. And the variety in real computation suggests that 
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the norm in rational performance is not one idealized 
super-device but “boundedness” of resources and powers. 
Matching this practical concern is a fundamental issue. In 
the complexity theory of space and time resources needed 
for computational tasks, we are really talking about the 
information in the world and how to process it. But this 
forces us to reflect on what is the information available 
to rational agents (Adriaans & van Benthem, 2008). And 
there is yet more to be learnt from the world of comput- 
ing. If we think of the behavior of rational agents as per- 
forming many tasks at once, just as networks of computers 
do, then there is a fundamental issue of architecture. How 
do the different components of the overall system pass 
information and cooperate (Gabbay, 1999)? 


3.2 Artificial Intelligence 

Moving closer to humans, computer science blends seam- 
lessly into AI. From the start, computers have been seen 
as a powerful model for human intelligence. In an inter- 
esting departure from the detailed internal analysis of 
computing by Turing machines, the famous Turing Test 
approaches intelligence in the tradition of measuring 
theoretical notions by external observable behavior (Tur- 
ing, 1950). It proposes that a computer achieves intelli- 
gence if an observer using natural language cannot tell 
that computer apart from a human by asking questions 
and engaging in conversation. Over the years, computers 
started to pass variants of the Turing Test, or other types 
of intelligent behavior. Actually, none of these are usually 
considered conclusive, as the criteria are a moving target 
(van Harmelen, Lifschitz, & Porter, 2008). Passing the 
test is dismissed as not a display of “real intelligence,” 
and then the demands are shifted a little further. But 
behavioral tests are crucial to judging human rationality 
as well. We seldom look inside people’s heads to moni- 
tor their considerations but rather observe their words 
and actions. 

A final intriguing feature of the Turing Test is its hybrid 
scenario where different types of agents, humans and 
machines, interact, presaging the reality of human-machine 
interactions in modern society. This scenario goes beyond 
classical emulation or competition concerns. How can 
societies of mixed agents, with different strengths and 
weaknesses, interact successfully (Wooldridge, 2002/2009)? 
The resulting diversity in agency is only beginning to 
be acknowledged more widely. Most paradigms in logic 
or philosophy assume that agents have similar abilities 
for reasoning, observing, and communicating, although 
their information and preferences may differ. Such unifor- 
mity assumptions underlie generic notions like “human” 
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or “rational actor,” and they may even seem to embody 
moral imperatives, like treating everyone equally qua rights 
and duties. If one accepts diversity, however, notions and 
theories concerning rationality must be rethought. 

The above trends in AI and computer science consider- 
ably extend the logical agenda for studying rationality. 
A major view of computation from the 1980s onward is 
that of behavior by agents (van Benthem, 2018), studied 
by merging ideas from computational and philosophical 
logic (Gabbay & Guenthner, 1983-; Gabbay, Hogger, & 
Robinson, 1993-1998). After explaining what is involved 
in such agency in section 4, concrete examples will be 
given in sections 5 and 6, showing logic at work in this 
modern setting. However, the logic-oriented agency 
approach has not gone unchallenged. In section 7, we 
will discuss the “high” versus “low rationality” competi- 
tion in understanding social agency and in section 8 the 
rise of nonrepresentational machine-learning techniques. 
Both come with a greater emphasis on probability, the 
other main formal paradigm for studies of rationality. We 
will end with an assessment of the current landscape of 
logico-computational approaches to rationality. 


4. From Machines to Rich Rational Agents 


4.1 A Conceptual Catalogue 

Let us first think of what agents can do in general. Human 
agents have a much wider range of rational activities 
than just reasoning from given data, that is, elucidat- 
ing what was already implicitly there. A rich dynamic 
information flow guides action. Agents constantly pick 
up new information from their environment by means 
of observation and communication, and they search 
their memory for information, too. Rationality is about 
picking up relevant information from whatever source is 
available, as much as reasoning, and that both in daily 
life and in science. 

However, even rich processing of correct informa- 
tion is just one dimension. Information can be less or 
more reliable, and agents do not just accumulate knowl- 
edge but also form beliefs that can be shown wrong by 
new information. Thus, robustly rational agents are not 
those who are always correct but those who learn from 
errors and have a talent for correcting themselves (Kelly, 
1996; Popper, 1963). Many facets of belief revision and 
learning are dealt with in chapter 5.2 by Rott; chap- 
ter 5.3 by Kern-Isberner, Skovgaard-Olsen, and Spohn; 
and chapter 5.4 by Gazzo Castafieda and Knauff (in this 
handbook). Logic is a major supplier of models here (van 
Benthem & Smets, 2015). 
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And rational agency does not stop here. Truly rational 
agents maintain a harmony between their information 
and beliefs with, on a par, their preferences, goals, or 
intentions. One can discuss which harmony is essential, 
whether maximizing expected utility (see chapter 8.2 
by Peterson, this handbook) or some other option: logic 
cannot, and should not, decide. Real agents may differ 
widely in their distance from classical decision-theoretic 
views (cf. chapter 8.3 by Gléckner, this handbook), but 
the crucial point of rationality remains maintaining 
a workable balance between information, goals, and 
actions. 

Summing up, a rational agent can gather information 
in a variety of ways, integrating observation, inference, 
and communication (van Benthem, 2011). In this pro- 
cess, the agent can form a rich variety of attitudes, from 
knowledge and belief to rejection or doubt. Also, the 
agent can function in environments where beliefs turn 
out wrong and learns from errors. And all this maintains 
a purpose, a balance between the agent’s goals and its 
information or actions. And finally, even this is not yet 
a full picture: rational agents display their skills in social 
interaction, a topic that will return. 


4.2 Multiagent Systems 

Bits and pieces of this richer notion of rational agency 
have long been studied by philosophers and logicians: 
witness the Handbook of Philosophical Logic (Gabbay & 
Guenthner, 1983-). In the 1980s, a congenial picture of 
agency emerged in computer science and AI, namely in 
the study of multiagent systems (Fagin, Halpern, Moses, & 
Vardi, 1995; Shoham & Leyton-Brown, 2008; Wooldridge, 
2010). This reflected a shift in thinking about comput- 
ing, from machines to agents with a behavior analyzed 
in terms of features normally ascribed to humans. This 
extends to autonomous systems. Robots investigate their 
environment with sensors, decide, and act in performing 
their tasks (Cardon & Itmi, 2016). Again, tools from phil- 
osophical logic make sense (cf. Brafman, Latombe, Moses, 
& Shoham, 1997, on epistemic specifications for real 
robots, whose sensors have a margin of error). But con- 
versely, notions from multiagent systems can be found 
in modern epistemology (Arl6-Costa, Hendricks, & van 
Benthem, 2016). For instance, robots acting on evidence 
of varying quality have inspired new models for evidence- 
based belief (van Benthem & Pacuit, 2011). 


4.3 Games 

There is a natural confluence here with one more disci- 
pline. Agents that acquire information, choose actions, 
and pursue goals are like players in games. Indeed, 


Johan van Benthem, Fenrong Liu, and Sonja Smets 


computer science has drawn closer to game theory (Nisan, 
Roughgarden, Tardos, & Vazirani, 2007), and logic, with 
its connections to games of argumentation (cf. chap- 
ter 5.5 by Hahn & Collins and chapter 5.6 by Woods, 
both in this handbook) and information seeking (Hin- 
tikka, 1973), is a natural partner. Indeed, epistemic game 
theory (see chapter 9.2 by Perea, this handbook) can be 
seen as a venture created by these contacts. 

Even with all this, no canonical view has crystallized 
yet of what a rational agent is and does that would be 
similar in elegance and fertility to that of a computing 
machine, let alone of a “universal rational agent” com- 
parable in its sweep to the universal Turing machine. In 
fact, the earlier recognition of diversity suggests a focus 
on different kinds of rational agents rather than unique- 
ness, changing standard assessments of performance. 
Is a rational agent someone who wields vast cognitive 
powers or someone doing their best with limited pow- 
ers? Are rational agents those who perform well against 
other rational agents or those who cope with a large 
bandwidth of types of agents in their environment? 


5. Logical Models of Rational Agency 


In recent decades, many features of rational agents have 
been studied by logicians. Information update and knowl- 
edge change occur in temporal logics of agency (Belnap, 
Perloff, & Xu, 2001; Fagin et al., 1995; Parikh & Ramanu- 
jam, 2003). Another paradigm is dynamic epistemic logic 
(Baltag, Moss, & Solecki, 1998; van Ditmarsch, van der 
Hoek, & Kooi, 2007), which models processes whereby 
agents form and modify representations of the informa- 
tion at their disposal. Such updates are not inferences, but 
they can be described just as precisely in logical terms. 

Example: Dynamics of information flow. In a simple 
two-party dialogue, agent 1, who is uncertain about 
the truth or falsity of p, asks “p?” A second agent then 
truthfully and publicly replies, “Yes.” Analyzing the 
information flow in the dialogue, the first agent’s ques- 
tion conveys that she doesn’t know the answer but also 
that she thinks the second agent, a fully reliable source, 
does know. The second agent’s answer conveys that 1’s 
assumption about 2’s knowledge was correct, and also, 
after 2’s answer, p is common knowledge in the group of 
these two agents. 4 

This mixture of knowledge of facts and knowledge 
about others is typical for communication. More com- 
plex scenarios, such as the famous Muddy Children 
puzzle (Fagin et al., 1995), illustrate how even truthful 
public announcements of ignorance following consecu- 
tive questions can gradually lead agents to knowledge. 
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A symbolic language capturing all of this has formulas 
[!g|Ky saying that the agent will know yw after a public 
event carrying the information that ¢ is true. A key law 
in this logic of public announcements is the equivalence 


[alky (o> kl[¢ly) 


This relates new knowledge after the event g happened 
to its “preencoding” before the event: the agent had 
conditional knowledge that the event !@ would result in 
the truth of wy. Such logical laws interchange dynamic 
operators for events and epistemic operators for atti- 
tudes of agents, a crucial ingredient in understanding 
information update and knowledge change. 

Logics of belief change under hard and soft infor- 
mation have been developed in the same style (cf. van 
Benthem & Smets, 2015), and other formal approaches, 
such as “AGM,” occur in chapter 5.2 by Rott (this hand- 
book). Learning fits well with belief revision: connec- 
tions between dynamic logics of knowledge and belief, 
on the one hand, and formal learning theory, on the 
other, are found in Baltag, Gierasimczuk, and Smets 
(2011). 

Example: Belief change. In the above two-party dialogue, 
now assume that agent 2 is not a fully reliable information 
source. Starting from the initial situation, in which agent 
1 believes neither p nor =p, the answer of 2 to her ques- 
tion can trigger 1 to change her mind and, possibly, to 
adopt a wrong belief. Yet how exactly she changes her 
mind depends on the trust that 1 has in 2 as an informa- 
tion source about p. If 2’s answer “Yes” is considered to 
be reliable but not infallible, the “belief upgrade” that it 
triggers can be more radical, inducing a strong belief in p, 
or more conservative, inducing a weak beliefinp. 4 

Again, this process obeys logical laws. A formal lan- 
guage now has constructs [fq] for effects of conserva- 
tive upgrades and [fq] for radical upgrades. This brings 
to light many principles of belief change. For instance, 
-K-9 [ft q|Bg@ says, for factual statements g, that the 
agent comes to believe that @ is the case, unless she 
already knew before the announcement that ¢ was false. 
In logical studies of learning, one studies iterations of 
such upgrades and analyzes how well they perform as a 
learning method. 

The study of key features of rationality in this style 
continues. Further aspects of purposeful rational behav- 
ior brought into the scope of logic include the manage- 
ment of current “issues” that guide inquiry for tasks 
at hand (cf. the “inquisitive logics” of Ciardelli, Groe- 
nendijk, & Roelofsen, 2019). 

Next, moving from informational tasks to agents’ 
preferences and goals, which determine how they 


189 


evaluate situations, Liu (2011) studies logics of pref- 
erence change, which is connected to goal dynamics. 
Preference dynamics shows similarities with deontic 
logics, describing what is obligatory and permitted for 
agents in environments where new commands change 
the moral ordering of situations and actions (Yamada, 
2008). 

Finally, rational agents balance their information 
with preferences and goals. Logics combining all these 
features occur in influential frameworks for multiagent 
systems such as BDI (Rao & Georgeff, 1991), inspired by 
Bratman (1987), describing how agency is driven by a 
balance of beliefs, desires, and intentions. But perhaps 
the most active research area where all these features 
of rational agents meet is in the logical study of strate- 
gic behavior and equilibria in games (cf. chapter 9.1 by 
Albert & Kliemt, this handbook). 

Example: Reasoning about extensive games. Consider a 
finite game with two players, A and E, and outcome-values 
written in the order A-value—E-value (see figure 2.6.1). 

Intuitively, the outcome (99, 99) seems best for both 
players—but that is not what the standard game solution 
algorithm of backward induction yields. Looking from 
the bottom to the top, if E is to play, she will choose left, 
and so, if A believes that E will make this choice, she 
herself will play left in the first round and end the game, 
with outcome (1, 0). 

Analyzing why this game might play out in this non- 
Pareto-optimal manner involves many notions, includ- 
ing players’ actions, beliefs, preferences, and plans. All 
of these have been studied by logicians, in different set- 
tings or as separate topics. For precise definitions of equi- 
libria in games and a survey of logical game analysis, see 
van Benthem and Klein (2019). 


0, 100 99,99 


Figure 2.6.1 
An extensive game. 
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All dynamic logics discussed here exemplify the 
earlier-mentioned tandem of algorithm and data in com- 
putation. The events that produce new information or 
new desires operate on well-chosen static models that 
support attitudes of knowledge, belief, preference, and 
the like. 

Digression: Nonclassical logics. There are also approaches 
folding all of the above activities under varieties of infer- 
ence, emphasizing departures from classical consequence 
to nonclassical, nonmonotonic logics (Horty, 2012) and 
resource-conscious substructural logics (Restall, 2000). 
For a comparison of the two methodologies, see van Ben- 
them (2019). 

Example: Nonmonotonic reasoning. Consequence in 
classical logic is monotonic; new premises do not invali- 
date earlier conclusions: if [= g~andT CA, then AF gq. In 
contrast, default inferences are defeasible (cf. chapter 5.2 
by Rott, this handbook): if I know that Tweety is a bird, 
I can conclude that Tweety can fly, yet with a further 
premise that Tweety is a penguin, it no longer follows 
that Tweety can fly. 4 

The field of nonmonotonic logic studies proper- 
ties of default reasoning. In contrast, dynamic logics of 
belief revision capture default phenomena on a classi- 
cal base, locating the nonmonotonicity in belief change 
rather than in changing the inference rules: I believed 
that Tweety can fly, but after an event !Penguin(Tweety), I 
have lost that belief. 

Discussion. The logical study of ever more aspects 
of agency aims at a not exclusively behavioral view 
of rationality by identifying key internal features and 
mechanisms. But combining logical and computational 
agendas does not make logical systems realistic software 
agents or human agents. Far more is needed for algo- 
rithms to work, and implementation requires further 
syntax. Recent studies mediate between semantic mod- 
els and syntactic representations for computing agents 
(Halpern & Régo, 2013; Lorini, 2018). 

Also, the development of ever richer models raises 
questions. Where is the boundary of agency, as more 
and more topics are taken on board, and what is “ratio- 
nal” about the activities so described? Are we describ- 
ing what agents actually do, or are these logical systems 
normative? A common view holds that logics of agency 
describe idealized laws that may or may not be fol- 
lowed by actual agents. This tension may be just what is 
needed. We cannot say, for instance, that belief revision 
leads to “correction” of earlier beliefs unless we have 
a norm for what is correct in the given circumstances. 
These are big issues that we cannot settle here but that 
permeate much of this handbook. 
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6. Rationality in Interactive Social Settings 


The modern study of agency reflects the fact that the par- 
adigm of computing today are distributed systems, not 
single machines. Likewise, multiagent systems put inter- 
acting individual agents at center stage and, at a next 
level, view groups themselves as actors, up to crowds 
or societies. This shifts the location of rationality from 
single agents to the quality of their interactions. And it 
broadens the focus from individual desires and actions 
to include emergent properties of the social system. 

The interactive perspective is not alien to logic. Ever 
since Antiquity, dialogue, argumentation, and debate 
have been paradigmatic scenarios, and the rich inter- 
face of logic and games has been noted already (Hodges 
& Vaananen, 2001/2019; van Benthem, 2014). A core 
topic in epistemic logic is the rational ability to reason 
about others, with iterated forms such as “Agent i knows 
that agent j knows that...” and analogues for belief 
and other attitudes (cf. chapter 5.5 by Hahn & Collins 
and chapter 5.6 by Woods, both in this handbook). This 
recursion to higher levels is widespread: we can even 
be afraid of fear, of fear of fear, and so on. The extent 
to which human agents truly display these abilities is 
studied in cognitive psychology under the heading of 
Theory of Mind (Isaac, Szymanik, & Verbrugge, 2014; 
Premack & Woodruff, 1978). Iterated knowledge is used 
in computer science in analyzing correctness and secu- 
rity of communication protocols (Fagin et al., 1995). 

But there is much more to social interaction than epis- 
temic reflection. Strategic action involves dependencies 
of one agent’s behavior on that of others or, better, on 
her expectations about the behavior of others (Aumann, 
1995). Here is a simple illustration, a computational task 
in an interactive setting. 

Example: Sabotage game. A Traveler in a graph moves 
along edges to reach some specified goal region. This 
graph-reachability problem is solvable in polynomial 
time. But now, there is a malevolent Demon who cancels 
an edge after each move Traveler makes. After that, Trav- 
eler goes along some still existing edge, andsoon. 4 

This “sabotage game” models search tasks under 
adverse circumstances and other social-informational 
scenarios. The solution complexity of the sabotage game 
jumps from polynomial time to complete for polynomial 
space. Logic helps determine who has a winning strategy 
in a given sabotage game by defining the basic challenge— 
response pattern, and it helps reason about general prop- 
erties of such games. This is one case where logic meets 
“gamifications” of agency scenarios, and logics can even 
be used to devise concrete new practical games, thus 
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becoming a tool of design as much as of analysis (van 
Benthem, 2014). 

With preference added, different notions of rationality 
have been investigated by logical means, such as those in 
game solution methods like backward induction, iterated 
removal of strictly dominated strategies (see chapter 9.1 
by Albert & Kliemt and chapter 9.2 by Perea, both in this 
handbook), or iterated regret minimization (Halpern & 
Pass, 2009/2012). The structure of strategies in them- 
selves is studied extensively at the interface of game the- 
ory, logic, and computer science (Brandenburger, 2014; 
van Benthem, Ghosh, & Verbrugge, 2015). Also, games 
influence computational logic: witness the “Boolean 
games” of Harrenstein, van der Hoek, Meyer, and Witte- 
veen (2001), where players can manipulate truth values 
of propositions toward achieving their goals. 


7. High and Low Rationality 


At this point, a challenge arises to the preceding analyses of 
individual and social rationality. Classical game theory has 
agents that deliberate and design complex strategies, and 
rich epistemic and dynamic logics of agency reflect this. 
However, in evolutionary game theory (see chapter 9.3 by 
Alexander, this handbook), poor agents, perhaps hardwired 
biological types (Maynard Smith, 1982), do just as well. 
Example: Evolutionary games. In a “Hawk-Dove” game, 
two individuals compete for a resource and can adopt 
either a Hawk ora Dove strategy. Hawks fight aggressively 
against other Hawks, in order to obtain the resource, until 
injury occurs and one retreats or, when facing a Dove, 
just take the resource, while a Dove retreats when facing 
a Hawk or shares the resource when facing a Dove. 4 
Game theory computes equilibria here, which are 
typically in mixed strategies. With repeated Hawk—Dove 
games, the appropriate notion is that of an “evolution- 
arily stable” strategy, and it can be shown that certain 
mixtures of Hawk-Dove populations are stable, when 
the value of obtaining the resource is greater than the 
cost associated with possible injury in a fight. One can 
think of these mixed strategies as complex behavior for 
individual reasoning agents but also as just percentages 
of a population consisting of two types of agents, each 
just doing what it does, perhaps for biological reasons. 
In the terminology of Skyrms (2010), the realm of 
complex reasoning players is that of “high rationality” 
and the realm of hardwired simple agents that of “low 
rationality.” Often the latter do as well as the former. For 
instance, a classical game-theoretic analysis might say 
that through some sophisticated Kantian or Rawlsian 
argument involving thinking about others, we all arrive 
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at the conclusion that we should live by the principles 
of morality, with the exception perhaps of a few free rid- 
ers. By contrast, a game-theoretic evolutionary stability 
argument may tell us that in the long run, a popula- 
tion with a certain proportion of simple law-abiders (the 
prey) and lawbreakers (the predators), who both cannot 
help being what they are, is stable. No reasoning need be 
involved at all; the morality is emergent system behav- 
ior. This influx from evolutionary game theory reinforces 
the message of distributed computing: a society of many 
simple agents can produce highly complex behavior. 

Here is one more scenario of emergent long-term 
complex behavior. 

Example: Limit behavior in social networks. The follow- 
ing network has an update rule that agents s (the nodes) 
adopt a belief p if p is held by all their neighbors (that 
is, all nodes with an arrow pointing to them from s). 
Applied iteratively, the following evolutions may occur 
with different initial situations (see figure 2.6.2). 

Case 1: Initial p = {1}. The second stage has p = © (no 
one believes p), and this remains the outcome ever after. 
Case 2: Initial p = {2}. The next successive stages are 
{3}, {4}, {2}; from this stage onward, the network activa- 
tion states loop. Case 3: Initial p = {1, 2}. The next stage 
is {3}, and we get an oscillation as before in case 2. Case 4: 
Initial p = {1, 2, 3}. We get {1, 3, 4}, {2, 4}, {2, 3}, {1, 3, 4}, 
and an oscillation starts here. 4 

Thus, network update dynamics can stabilize in a single 
state (case 1) or oscillate in loops. Sometimes, successive 
models in a loop are isomorphic (cases 2 and 3), and some- 
times the loop runs through different nonisomorphic net- 
work configurations (case 4). In infinite networks, an even 
further option is divergence toward ever different config- 
urations. In all these scenarios, no logic seems involved 
in belief formation: behavior arises from agent types (the 
update rule) and the global structure of the network. 

To put the challenge starkly, perhaps complex, logic- 
based rationality is not necessary to understand the behav- 
ior of human and artificial agents? But things are more 
complicated. In daily life, we think carefully in the “high” 
style about certain issues, but given our limited resources, 
we just follow, “low” style, our neighbors on perhaps the 
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Figure 2.6.2 
A social influence network. 
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majority of issues. This mixture calls for explanation, and 
current investigations are charting its details. 

Combined high-low scenarios. Liu, Seligman, and Girard 
(2014) study agents in social networks that follow their 
neighbors’ preferences, beliefs, or behavior via rules like 
following the majority or some other threshold, reflect- 
ing different agent types. The resulting diffusion process 
models the spread of fashions and new ideas. But agents 
still have epistemic states, and these can change dynam- 
ically as before, as described in an “epistemic friendship 
logic.” In this setting, among other things, a logical 
characterization can be given of conditions for stabiliza- 
tion of agent’s beliefs, thus predicting long-term system 
behavior. This framework combines ideas from sociology 
(Friedkin, 1998) with epistemic logic, adding an essen- 
tial element to the earlier logical models of agency: the 
structure of the social network (see also chapter 10.1 by 
Dietrich & Spiekermann, this handbook). For a conge- 
nial study in another logical framework, see Xue (2017). 

Other combined scenarios that have been studied 
include groups of individually rational agents who rea- 
son toward a common decision. Two things can happen: 
individual agents can enhance each other’s reasoning 
power and bring about a higher level of group rational- 
ity, surpassing that of each individual agent. But groups 
may also get locked in irrational behavior. Whether the 
one will happen or the other is investigated in Baltag, 
Boddy, and Smets (2018), in terms of differences in inter- 
ests and abilities between agents. One striking conclu- 
sion is that irrational group behavior is often not caused 
by irrational behavior of individual agents but by mis- 
alignments of their interests. 

Other social phenomena that have been studied by 
logical techniques are informational cascades (Bikh- 
chandani, Hirshleifer, & Welch, 1992), where a sequence 
of individual agents follows the decisions of their pre- 
decessors while ignoring their own private evidence. 
Baltag, Christoff, Hansen, and Smets (2013) ask whether 
individual rational agents, who use all their higher- 
order reasoning power, can stop a cascade from happen- 
ing. The answer, surprisingly, is “no,” and this fact can 
be proved by logical techniques that track information 
updates. However, the protocol regulating the agents’ 
strategies matters: when agents have total communica- 
tion and sharing of evidence, cascades can be stopped. 

The preceding examples show how prima facie ide- 
ological differences between high and low rationality 
turn into a deeper study of how the two interface. This is 
logic at work at the ground level of agent activity. How- 
ever, there is also a second, more methodological contact 
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between the two sides: logic can analyze the structure 
of the dynamical systems theory underlying most low- 
rationality approaches and find patterns there, usually 
amounting to high-level qualitative descriptions of sys- 
tem behavior. Explorations in this direction include Kre- 
mer and Mints (2005) and Klein and Rendsvig (2017). 


8. Machine Learning and Probability 


In addition to the preceding tensions, the contemporary 
world of computing and AI offers a new challenge to 
logic-based views of rationality. 


8.1 Machine Learning 

Machine learning (Kelleher, Mac Namee, & D’Arcy, 2015) 
works well on large data sets, outperforming symbolic 
approaches, which tend to have problems of scalability. 
For instance, in supervised learning, a neural network is 
constructed, consisting of nodes with adjustable thresh- 
olds and links of adjustable strengths between nodes. 
Each setting for all of these produces an activity in the 
output layer given an input to the initial layer of the net- 
work. A cost function measures the distance of the current 
outputs to the desired ones on the training inputs. The 
network can then, by well-known techniques such as gra- 
dient descent (Russell & Norvig, 1994), adjust its weights 
and thresholds in the direction of lowering the cost func- 
tion. In the end, stable optima in network activation are 
reached that work very well in new cases outside of the 
training set, in many computational and cognitive tasks. 
These networks, related to spin glass models in physics 
(Nishimori, 2001), use general statistical methods rather 
than specifically human agent features. 

Neural networks in machine learning do not have 
any features immediately corresponding to classical 
logical models. There is no language and no represen- 
tation, and the dynamic operations of the network do 
not reflect logical operations in any obvious manner. 
Also, very different stable states of the network result- 
ing from training sessions can perform the same tasks, 
and invariants are hard to detect. Thus, whereas low- 
rationality methods raised the question whether logical 
analysis was necessary, machine-learning methods raise 
the question whether logical analysis is even possible. 

It is far too early to adjudicate this debate. But here, 
too, there are some promising developments toward coop- 
eration rather than antagonism. Integrating inference as 
used in neural networks and learning systems with sym- 
bolic reasoning is an active area of research (Baggio, van 
Lambalgen, & Hagoort, 2015; Balkenius & Gardenfors, 
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2016; Leitgeb, 2004). “Explainable AI” seeks humanly 
intelligible qualitative patterns behind machine-learning 
systems, with topics such as causal reasoning (Halpern, 
2016; Pearl, 2000; van Rooij & Schulz, 2019; see also chap- 
ter 7.1 by Pearl, this handbook), conditional logics as a 
way of classifying types of machine learning (Ibeling & 
Icard, 2018), or extracting logical inference modules from 
actual neural networks (Geiger, Icard, Lu, & Potts, 2021), 
while there is also a trend toward finding joint perspec- 
tives on learning in itself. 

But also, recall a distinction made at the start of this 
chapter. If logic is only seen as a direct model for activities 
of reasoning or information update, other frameworks 
look like competitors. To some, the only question under 
debate is then whether logic can enhance such frame- 
works in terms of representation or computation. But in 
the more foundational sense of logic as an analysis of the 
structure of theories of computation and agency, even 
machine learning works on spaces with logical structure 
that can be described in logical terms (Leitgeb, 2017), 
and a meeting of the minds seems entirely feasible. 


8.2 Probability 
Continuing with methodological issues, here is one 
final contrast. A conspicuous feature of most studies of 
agency is the extensive use of probabilistic methods, a 
quantitative paradigm often seen as being at odds with 
qualitative logical analysis. Probability underlies many 
computational systems; it lies at the heart of game the- 
ory and dynamical systems theory, and in epistemology, 
probabilistic styles of analysis are at least as widespread as 
logic-based ones (see chapter 4.1 by Hajek & Staffel and 
chapter 4.7 by Dubois & Prade, both in this handbook). 
The fruitful issue here is again one of combination. 
Qualitative and quantitative approaches naturally coex- 
ist, and the issue is just how. For instance, epistemic and 
doxastic logics, both static and dynamic, model uncer- 
tainty in terms of ranges of options (Adriaans & van 
Benthem, 2008), whereas Bayesian epistemology uses 
updates of probability functions (Bovens & Hartmann, 
2003; Talbott, 2016). The compatibility of the two per- 
spectives shows in combined systems (Halpern, 2003) 
that reason about both ontic and epistemic uncertainty, 
bringing together logic-based approaches with proba- 
bilistic conditioning. Other uses of probability concern 
action rather than information: witness the mixed strat- 
egies in game theory (for a logical perspective, see van 
Benthem & Klein, 2019). But there are also quite dif- 
ferent interfaces of logic and probability, for instance, 
in the data-oriented parsing (DOP) architecture of Bod, 
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Scha, and Sima’an (2003) and Bod (2008), which com- 
bines classical rule-based models of language and reason- 
ing with probabilistic pattern recognition in a memory 
of earlier performance. Finally, the foundations of prob- 
ability were still close to logic in the work of Boole and 
de Finetti, and various strands of research link the two 
realms in new ways. Harrison-Trainor, Holliday, and 
Icard (2018) study low-complexity qualitative reasoning 
systems that admit of introducing probability measures, 
while Leitgeb (2017) derives qualitative notions of belief 
from richer probabilistic models. 

There are many further philosophical and technical 
issues to be explored at this rich and growing set of inter- 
faces that we cannot cover in this chapter; the reader 
is referred to Spohn (2012) and chapter 5.3 by Kern- 
Isberner, Skovgaard-Olsen, and Spohn (this handbook). 


9. Conclusion 


This chapter has presented broad perspectives from logic 
and computation on rational agency. These ranged from 
high-level foundational insights into information and 
proof to specific studies of various abilities of information- 
and goal-driven agents. A rational agent, in this light, is 
a reasoner, information-processor, concept-crafter, and 
purpose-seeker: fallible but talented. Is it also a human 
cognitive agent? On connecting logic and computation 
to cognitive reality, we refer to Stenning and van Lambal- 
gen (2008). 

The main thrust of a logical approach as we see it 
is theoretical, but the deep entanglement of logic and 
computation over the past century has added practi- 
cal dimensions. Rationality as studied here can be pro- 
grammed and put into intelligent systems, even though 
the path to feasibility is not easy or trivial. It is this very 
distance that allows logical theories to also be norma- 
tive, providing an essential tension between the real and 
the ideal in the study of rational behavior, which keeps 
sparking further investigation. 

We have not hidden the fact that the classical logico- 
computational paradigm faces challenges, coming from 
probability theory, dynamical systems theory, and machine 
learning. But we think this is all to the good, since these 
challenges suggest new interface topics of interest to all. 

Finally, it should be clear that we have not claimed 
that logic is the only game in town. Neither is compu- 
tation. The approach surveyed in this chapter does not 
hold the unique key to understanding the rich phenom- 
enon of rationality, but it does offer one valid and illu- 
minating perspective. 
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Section 3 Deductive Reasoning 


3.1 Propositional and First-Order Logic 
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Summary 


This chapter addresses the question how (if at all) prop- 
ositional logic (PL) and first-order logic (FOL) relate 
to epistemic rationality.' Rationality, it is often held, 
demands that our attitudes cohere in particular ways. 
Logic is often invoked as a source of such coherence 
requirements when it comes to belief: an ideally ratio- 
nal agent’s beliefs are consistent and closed under logical 
consequence (i.e., the logical consequences of the agent’s 
beliefs are also believed). However, this traditional pic- 
ture has been challenged from various quarters. I begin 
by briefly reviewing the key concepts involved in PL and 
FOL. I then critically examine two distinct approaches 
to justifying logic-based requirements of rationality. The 
first lays down a set of desiderata codifying our intuitions 
and then seeks to formulate a principle articulating the 
link between logic and rational belief that satisfies them. 
The second starts by identifying our most fundamental 
epistemic aim and seeks to derive requirements of ratio- 
nality based on their ability to promote this aim. 


1. Logic and Rationality 


Rational belief and correct belief are not the same thing. 
I may rationally believe a falsehood (for example, when 
the evidence available to me is misleading). If correct- 
ness for beliefs is truth, my belief may be correct though 
irrational (as when I engage in wishful thinking and, 
merely by a fluke, form a true belief). Part of what it 
means for beliefs to be rational is for them to cohere 
with the evidence. Another part is for them to cohere 
with each other.’ Logic is often thought to be connected 
to the second sense of rationality. Logic is concerned 
with the relationships between the truth-values of differ- 
ent propositions. It tells us which propositions can and 
cannot be jointly true by dint of certain of their struc- 
tural properties. For example, a chief concern of logic is 
logical consequence. If a proposition C is a logical con- 
sequence of a set of propositions, all of which I believe, 


it is, in a specific sense, impossible for my beliefs to be 
true without C also being true. Conversely, if I know C 
to be false, I thereby know that my antecedent beliefs 
cannot all be true. Relatedly, if my beliefs are inconsis- 
tent (and I recognize them to be so), then, merely by 
virtue of the logical structure of my beliefs’ contents, I 
am in a position to know that I must be mistaken with 
respect to at least one of them—my representation of 
the world is not merely false, but there is no logically 
possible world that makes all of my beliefs true. Logical 
coherence, then, is typically a necessary (though gener- 
ally insufficient) condition for the truth of one’s belief. 
And while I may not be rationally criticizable for having 
incorrect beliefs, many maintain that if my beliefs fail to 
be logically coherent, I am less than ideally rational. It 
is in this sense that logic is thought to offer up require- 
ments of rationality.’ 

The relation between logic and epistemic rationality 
relies on our assumption that the entities with which 
logic is concerned are also (or are systematically related 
to) the contents of our rationally evaluable mental 
states. I use “proposition” to designate the type of entity 
that constitutes both the premises and the conclusions 
of logical arguments, on the one hand, and the contents 
of propositional attitudes, on the other. I assume there 
is a type of entity capable of performing both of these 
roles. Furthermore, as my talk of structure in the fore- 
going makes plain, I assume that propositions are (logi- 
cally) structured entities. That said, what I go on to say 
is compatible with other, nonpropositional accounts of 
truthbearers. 

The aim of this chapter is to provide a critical survey 
of the discussion over whether logic really does give rise 
to norms of epistemic rationality. More specifically, I ask 
whether classical propositional logic (PL) and its exten- 
sion, first-order logic (FOL; also called “[first-order] pred- 
icate logic”), can be said to do so. The force of restricting 
ourselves to these logics comes to this. Logic, we said, is 
concerned with possible truth-value distributions across 
propositions based on their logically relevant structural 
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features or logical form. The restriction to these particu- 
lar logics is relevant in that different logics are sensitive 
to different structural features. The richer a logic’s lan- 
guage, the more logical structure it is able to discern and 
hence the more rationally constraining it will be. 

Another elementary but important observation is that 
PL and FOL are not themselves explicitly about rational 
belief formation and revision—their logical vocabulary 
contains no symbols that are naturally interpreted as des- 
ignating doxastic states. In this they contrast with more 
sophisticated formalisms such as belief revision theory 
(Alchourr6n, Gardenfors, & Makinson, 1985), ranking 
theory (Spohn, 2012), and various Bayesian approaches 
whose express aim is to model the dynamics of belief 
management. The existence of these theories does not, 
however, render the present question nugatory. On the 
contrary, the fact that such theories incorporate implicit 
assumptions about the relationship between logic and 
rational belief renders the exploration of these founda- 
tional questions all the more urgent. 

The chapter is structured as follows. The first two 
sections provide brief reviews of PL and FOL. Section 4 
offers an overview of recent attempts at articulating the 
relationship between PL and FOL and rational belief 
as well as criticisms that have been leveled at such 
attempts. Section 5, finally, explores the relation of logi- 
cal requirements of rationality to recent work in formal 
epistemology. 


2. Propositional Logic 


For our purposes, we can conceive of a logic as a formal 
language along with a semantic (model-theoretic) and 
a syntactic (proof-theoretic) relation of logical conse- 
quence. When it comes to formal languages, the sym- 
bols making up the alphabet are usually divided into 
three separate categories: 


1. Descriptive symbols 
2. Logical symbols 
3. Auxiliary symbols 


Descriptive symbols, intuitively, are the expressions of 
the language that have variable semantic values and 
so may be used to express truth-evaluable claims about 
the world. PL treats propositions as atoms, that is, PL 
is blind to the internal structural features of proposi- 
tions. For example, it is insensitive to the distinction 
between propositions expressed by sentences involv- 
ing only a unary predicate and singular term (“Rachel is 
rational”) and sentences composed of binary predicates 
and two singular terms (“Rachel is taller than Steve”). 
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Consequently, the only descriptive symbols contained 
in PL’s language are formulas that serve to represent 
propositions (as opposed to expressions serving to express 
subpropositional content). PL distinguishes between 
atomic propositions, whose logical form cannot be fur- 
ther analyzed (using the resources of PL) and which are 
represented by atomic formulas, and complex ones, rep- 
resented by complex formulas. Extending the chemistry 
metaphor, complex formulas are molecular compound 
formulas held together by the bonds that are logical 
connectives. The logical connectives are the following 
unary or binary operators (their approximate English cog- 
nates are given in brackets): “7” (“not”), “a” (“and”), 
“v" (“or), “—o" (Gf..., then...”), and “<” (“if and 
only if”). The only auxiliary symbols are parentheses, 
which are needed to avoid ambiguity among our expres- 
sions (e.g., to distinguish (A a(BvC)) and ((An B)vC)). 
In summary, the language contains 


1. propositional variables: Y = {pi, p2, ps, . 
tice, we typically use “p,” “q,” “1,” .. .); 


. .} (in prac- 


2. logical connectives (or logical constants): =, a, v, >, 
<; and 
3. auxiliary symbols: (, ). 


This alphabet constitutes the language of PL, Zp. Our 

next order of business is to specify a grammar for our 

language. Call a grammatically correct expression in our 

formal language a formula. We now define the set of for- 

mulas of PL, Fp, by means of the following inductive 

definition: 

1. If AGP, then A EF py. 

2. If A €Fp,, then so is 7A. 

3. If AG Fp, and BEF», then so are (AaB), (AvB), 
(A — B), and (A = B). 

4. Only such strings of symbols of our alphabet as can 


be constructed in accordance with rules 1 to 3 are 
formulas. 


It is a direct consequence that every formula has a set of 
immediate subformulas (ISFs): 


- IfA ©, then ISE(A) = ©. 
* IfA = 7B, then ISF(A) = {B}. 


* If A =(B*C) (where* €{a,v,—,<}), then ISF(A) = 
{B, C}. 


2.1 Semantics of PL 

The semantics of PL mirrors its syntax in the following 
sense. The truth-value of a complex formula is determined 
by the truth-values of its immediate subformulas and 
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the logical connective governing it. Logical connectives 
are unary (negation) or binary truth functions from the 
truth-values of the subformula(s) to the truth-value of the 
complex formula to be evaluated. The atomic formulas, by 
contrast, are assigned their truth-values directly, by way of 
an interpretation. An interpretation in PL is a function 3 
whose domain is the set Y of propositional variables and 
whose codomain is the set of {t, f} of truth-values: 


3: A= tt, f}. 


Our grammar guarantees that every complex formula can 
be broken down uniquely into subformulas of decreasing 
complexity until the process bottoms out in the propo- 
sitional variables figuring within it. Consequently, the 
truth-value of a complex formula too will ultimately 
depend on the truth-values assigned to the proposi- 
tional variables it contains and on the particular mode 
of composition of the formula. Our aim now is to use 
this insight to extend each interpretation function into 
a corresponding valuation function that assigns to each 
formula a truth-value. More precisely, every interpreta- 
tion 3 in PL determines a unique corresponding valua- 
tion Bs (as can easily be proved). 

A valuation in PL (relative to an interpretation 3), 
then, is a function Bs: Fp, — {t, f} that satisfies the fol- 
lowing semantic rules: 


. B(pi) = t iff 3(p) = t, 
. By(7A) = t iff Ba(A) =f, 

. By((A n B)) = t iff Bo(A) = t and Bs(B) = t, 
. By((A v B)) = t iff Bs(A) = t or Bs(B) = t, 

. By((A > B)) = t iff Ba(A) = f or B(B) = t, 
. By((A + B)) = t iff Bs(A) = Ba(B). 


non FF WN 


Clauses 1 to 6 are also called “semantic rules.” Rule 1 
says that propositional variables are to be evaluated via 
Bs; just as the underlying interpretation 3 dictates. In 
rules 2 through 6, complex decomposable formulas are 
evaluated in conformity with the meanings of the logi- 
cal connectives. 

With this definition in place, we can now define the 
central concepts of consequence, validity, and consistency: 


1. For all formulas A;,..., A, and B of Fp: B is a logi- 
cal consequence in PL of Ai,..., An (in symbols: 
Ai,..-, An Fer B) iff for all interpretations 3 with 
By(Ai) =t,..., Bs(An) = t, it holds that B3(B) = t. 

., A, “. B of PL is valid in PL 


2. An argument form Aj, .. 
iff Ai,..., An Fe. B. 

3. A set of formulas {Ai,..., An} is consistent in PL iff 
there exists an interpretation 3 such that B3(Ai) = 
t,..., Bg(A,) = t. 
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It is not hard to see that the concepts of logical conse- 
quence and consistency are intimately related: 


* A set of formulas {Aj,.. 
not consistent) in PL iff A;,.. 


., An, B} is inconsistent (i.e., 
.) An Epp =B. 


To see this, observe that A;,..., A, Fp, 7B iff there exists 
no interpretation 3 such that B3(A;)) =t fori=1,...,n 
and %3(-B) = f. But Bs(-B) = f iff Bs(B) = t. Hence, Ai,..., 
A, Fp. 7B iff there exists no interpretation 3 such that 
Gx(A) =t fori=1,..., nm and B3(B) =t, but that is tanta- 
mount to saying that {Ai,..., An, B} is inconsistent. 


2.2 Natural Deduction for PL 

Let us now turn to the syntactic or proof-theoretic ana- 
logues of these notions. The proof system introduced here 
is a variant of natural deduction. It originates with Ger- 
hard Gentzen (1934/1969) and was designed as an alter- 
native to existing Frege—Hilbert axiomatic systems, with 
the aim of representing deductive reasoning more closely. 
“TEA” is to be read as “The formula A is derivable in our 
system from the set of formulas [.” For the most part, 
each connective is characterized in terms of a pair of rules. 
The introduction rules (. . . -I) state the conditions under 
which a proposition with the connective in question fig- 
uring as its main connective can be deduced; the elimi- 
nation rules (...-E) state the deductive consequences of 
having derived such a proposition. The system, which we 
refer to as NDp, is constituted by the following deduc- 
tive rules: 


HRA TFB TFA AA, 
T,UT,FAaB TEA, 
TEA, 


vg PitAVB Ty, AFC Ty, BEC 


v-I 


TFA, VA, T, UT, VE,FC 
a T, AFB - IT, FA I,FA-B 
TrA>B I, UI,FB 
,DAFL iu A TI, 1A 
TFAA TTUI,FL 
ECQ DNE. =————__= 
TFA TFA 


where, in (A-E) and (v-I), i= 1 ori = 2. 

“1” denotes a contradictory proposition. “ECQ” abbre- 
viates “ex contradictione [sequitur] quodlibet” (i.e., “from 
a contradiction anything follows”). “DNE” abbreviates 
“double negation elimination.” We can then say that 


* forall formulas Aj, ...,A, and B of Fp,: B is derivable from 
Ai,...,Anin NDp, (in symbols: Ai,..., An np,, B) 
just in case there is a derivation employing only NDp. 
rules and relying solely on premises Ai,..., An. 


NDp, can be proved to be sound: 
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Soundness: If Ai,..., An +p, B, then Ai, ..., An Fe. B, 
and complete with respect to PL: 
Completeness: If Ai, ..., An Fp. B, then Ai, ... , An Fp), B. 


3. First-Order Logic 


The language of FOL is expressively richer than that 
of PL, which manifests itself in two principal respects: 
(1) FOL is sensitive to logically relevant subsentential 
structure, and (2) FOL is able to represent quantified sen- 
tences such as “All donkeys have soft noses” or “Some 
people annoy everybody.” Since PL lacked the expres- 
sive resources to represent such sentences, it had to treat 
the propositions so expressed as unanalyzable. Conse- 
quently, PL is blind to the validity even of simple syl- 
logisms such as 


All donkeys have soft noses. Camillo is a donkey. There- 
fore, Camillo has a soft nose. 


FOL allows us to capture not only arguments such as 
these but also propositions involving multiple general- 
ity, where several quantifiers interact, as in our previous 
example “Some people annoy everybody.” 

Let us turn to the vocabulary of the language of 
FOL, Frou: 


1. Descriptive symbols: 
(a) Countably many individual constants: a), do, d3,... 


(b) Countably many predicate symbols: Pi”, Py, Py, .. . 
(for all arities n = 1) 


2. Logical symbols: 
(a) Connectives: 7, a, v, >=, <= 
(b) Quantifiers: 4 (“there is”/“some”), V (“all”/“every”) 


(c) Countably many individual variables: x1, x2, x3, ... 
3. Auxiliary symbols: 

(a) () 

(b), 


Different applications of FOL will necessitate different 
choices of descriptive symbols. Each choice determines 
a specific language of first-order logic. What is com- 
mon to all such languages are the logical symbols. The 
propositions expressed by “Camillo is a donkey” and 
“All donkeys have soft noses” can be represented by the 
formulas “D(c),” where “c” is a constant—comparable to 
a proper name—that denotes Camillo, “D” is the predi- 
cate “is a donkey,” and “S” is the predicate “has a soft 
nose.” The latter sentence can then be represented by 
“Wx (D(x) = S(x)),” where “Vx...” is to be read as “for 
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all x,...”; roughly, the sentence can be paraphrased as 
“For all things x, if x is a donkey, then x has a soft nose.” 

In the following, I use lowercase letters “t,” “c,” “ 
as meta-variables for singular terms, constants, and vari- 
ables, respectively. Note that for all ¢, tis a singular term 
in Yroxr iff t is an individual constant or an individual 
variable. We are now in a position to define the set of 
formulas of FOL, Fron: 


y” 


If P” is an n-ary predicate and t,,...., t, are singular terms, 
then P"(t, ... , t,) © Frot. These are the atomic formu- 
las of Fron. 

If AE Fro, then so is 7A. 

If AG Fro. and BEFyo., then so are (AaB), (AvB), 
(A > B), and (A = B). 

If A €Fyor and v an individual variable, then 4vA € Fro. 

If A€ Fro. and v an individual variable, then VVA € Fro. 

Only strings of symbols constructed in conformity with 
the clauses above qualify as well-formed formulas (wffs). 


We can again define the notion of an immediate subfor- 
mula in FOL: 


* If A is atomic, then ISF(A) = 2. 

+ IfA =-B, then ISF(A) = {B}. 

* IfA =(B:C) (where: €{a, v, =, <}), then ISF(A) = {B,C}. 
¢ If A = 4vB, then ISF(A) = {B}. 

* If A = VvB, then ISF(A) = {B}. 


3.1 Semantics for FOL 

The semantics for FOL takes a slightly different form due 
to the fact that we need an apparatus to assign semantic 
values to subsentential expressions. Moreover, we must 
specify a domain of objects for our quantifiers to range 
over. 


+ An interpretation, 3, in FOL is a pair (D, y) where 


1. Dis a nonempty set of objects, 
. mis an interpretation function such that 


2 

3. for all individual constants c, g(c) ED, 

4. for all n-ary predicates P", g(P”) CD", where D" is 
the n-fold Cartesian product of D, that is, the set of 
n-tuples of elements of D. 


Consider the atomic formula P(x). Suppose we put 
D = N (where N is the set of natural numbers) and interpret 
P by letting ¢(P) be the set of prime numbers. The formula 
P(x) cannot be said to be true or false, because we do not 
know what x refers to. We could prefix the formula with a 
quantificational expression as in 4x P(x), in which case the 
variable is said to be “bound” (as opposed to “free”) and 
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the formula does obtain a truth-value: it is true because the 
set of prime numbers is nonempty. Alternatively, we could 
replace x by a constant c, which we might interpret by set- 
ting yc) = 28. The result, P(c), again has a truth-value (it is 
false because 28 is no prime number). Individual variables 
thus function somewhat like demonstratives (e.g., “that”): 
without the benefit of appropriate contextual informa- 
tion, we cannot determine their referent. But here is the 
challenge. As in the case of PL, we want to explain the 
truth-value of an FOL formula as a function of the truth- 
values of its immediate subformulas. But to account for the 
truth-value of, say, 4x P(x), we must then first account for 
“the” truth-value of P(x). To handle such cases, we need the 
notion of a variable assignment. A variable assignment o 
relative to a domain D is a function that assigns to each 
variable an arbitrary referent in D. More precisely: 


* A variable assignment o under an interpretation 
S&S = (D, ¢) is a function that assigns to every individual 
variable v an element d of D. 


In our example, the question as to the truth-value of 
dx P(x) (relative to the above interpretation) turns on 
the question whether there exists a o such that o(x) 
is a prime number. That is, we effectively account for 
quantification by appeal to quantification over variable 
assignments in the metalanguage. 

For convenience sake, let us merge our variable 
assignments with our interpretation function. That is, 
given an interpretation 3 = (D,q), the interpretation 
function @ and any variable assignment o (under 3) can 
be folded into a single function, g, which now covers 
all singular terms—individual constants and individual 
variables: 


+ Let 3 = (D,¢) bean interpretation in FOL and o a vari- 
able assignment under 3; then gq, is defined as follows: 


1. for all individual constants c: @(c) = g(c), and 
2. for all individual variables v: @.(v) = o(v). 


We have now outlined the machinery needed for 
determining whether a formula—any formula, includ- 
ing those with free variables—is true or false relative to a 
variable assignment. Just as in the case of propositional 
logic, what we are after is a function that assigns to every 
formula a truth-value on the basis of an interpretation 
and a variable assignment under that interpretation. We 
will again use y, to designate that function. So, @,(A) des- 
ignates the truth-value of the formula A relative to the 
interpretation 3 = (D, ~) and given the variable assign- 
ment o. It is important to be clear about the double role 
the function q, is playing here. In the last paragraph, 
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played the role of a function that took singular terms— 
individual constants and individual variables—as inputs. 
Now we are charging it with the further task of taking 
formulas as inputs and outputting truth-values (rela- 
tive to the interpretation and the variable assignment 
in question). In practice, the context will make it abun- 
dantly clear whether the function q, is being applied to a 
singular term or to a formula. 

With this we are in a position to define valuation 
functions for first-order logic: 


* Let 3 = (D, @) be an interpretation in first-order logic 
and oa variable assignment under S. A valuation in 
first-order logic (relative to 8 and o) is a function 
defined for all singular terms (as we explained in the 
previous section), which moreover assigns to every 
formula A of Lro, of a given language of first-order 
logic the value t or f so as to satisfy the following 
semantic rules: 


gAP(h,..., t)) = t iff (@(h), .. 
@o(7A) = t iff gfA) = f, 

po((A a B)) = t iff po(A) = p(B) = t, 

@o((A v B)) = t iff @(A) = t or @(B) = t, 

po((A > B)) = t iff @o(A) = f or @(B) = t, 

po((A <> B)) = t iff yA) = go(B), 

gAVVA) = tiff for all variable assignments o’ under 
%, if o' is a v-variant of o, then (A) = t, 

8. @(AVA) = t iff there is a variable assignment o' 
under 3 such that o’ is a v-variant of o and @;(A) = t. 


-) Go(tn)) EG(P"),° 
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If 3 = (D,~) and (A) = t, we say that A is true relative 
to the interpretation 3 and the variable assignment o 
under S (similarly for falsity). 

Of course, to complete our definition, we must still 
define the notion of a v-variant: 


* Let o, o’ be variable assignments under an interpreta- 
tion 3. Then o’ is a v-variant of o iff for all individual 
variables v’ # v it is the case that o'(v’) = o(v’). 


Hence, if o' is a v-variant of o, then o' agrees with o 
on all individual variables except possibly with respect 
to v, where the two may (but need not) diverge. It fol- 
lows that every variable assignment is a v-variant of itself 
(with respect to any individual variable v). 

We are now in a position to define our key logical 
notions for FOL: 


* For all formulas Aj, ..., An, B: Bis a logical consequence 
of Ai,...,An(Ai,..., An Fror B) iff for all interpreta- 
tions 3 = (D, g) and all variable assignments o under 
S, if po(A1) =t,..., @o(An) = t, then ¢(B) = t. 
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* A set of formulas {Ai,..., An} is consistent in FOL iff 
there exists an interpretation 3 = (D,q@) and a vari- 
able assignment o under 3 such that (Ai) =t,..., 


Po(An) =f. 


3.2 Natural Deduction for FOL 
Our system of natural deduction NDp, can be extended 
into a system for FOL, NDro1, by adding the following 
two pairs of rules for the quantifiers. 

In the application of 4-E, one must ensure that a does 
not occur in 4x A(x), in T2, or in B. 


_ TFA) agate AxA(x) T,, A(a)-B 
Th Ax A(x) TUT,FB 
rr T+ A(a) = TF VxA(x) 
TE VXxA(xX) TE A(t) 


In V-E, every free occurrence of x in A is replaced by t. 
In the application of V-I, one must ensure that a does not 
occur inl, and x must be uniformly substituted for a and 
must not be bound by any other quantifier in A. If a were 
allowed to occur among the hypotheses T upon which 
the conclusion depends, the following would be possible: 


_ AFA) 
A(t) F Vx A(x) 


Similar examples can readily be given for the remaining 
restrictions. As in the case of PL, FOL can be shown to be 
sound and complete with respect to NDrou.° 


4. Logic and Rationality 


Having thus reminded ourselves of the key concepts and 
features of PL and FOL, let us return to the supposed 
connection between logic and rationality we sketched 
in the introduction.’ How exactly do the laws of logic 
express constraints on rational belief? 

We do sometimes talk as if logic were itself a theory 
of rational belief. For instance, the rules of our natu- 
ral deduction systems are typically referred to as “infer- 
ence rules” and are read accordingly (e.g., “From A and 
B, infer A» B”). We already noted above that such talk 
cannot be taken literally. According to Gilbert Har- 
man (1984, 1986), our talk of “inference rules” and the 
underlying identification of logic and rational belief 
formation is simply a category mistake. Logic is one 
thing; a theory of epistemic rationality is quite another. 
Logic has an abstract subject matter; it concerns itself 
with certain properties and relations of (sets of) prop- 
ositions. A theory of rationality, by contrast, aims to 
give a normative account of how we should form and 
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revise our beliefs. Once we recognize the fundamental 
difference between these two theoretical enterprises, 
according to Harman, we realize that an explanatory 
gap separates them. Harman is doubtful that there is a 
substantive and informative story to be told as to how 
that gap might be closed. 

But perhaps this is too quick. Even if deductive logic 
and a theory of rationality do not come to the same 
thing, there may still be an interesting normative con- 
nection between the two. Following John MacFarlane 
(2004), let us call a general principle that seeks to articu- 
late such a relation between logic and norms governing 
belief, a bridge principle. Schematically, a bridge principle 
can be represented as follows: 


(*) IfA1,...,AnEC, then N(a(Ai),... 


, a(An), B(C)). 


Such principles take the form of a material conditional the 
antecedent of which expresses an instance of logical con- 
sequence and the consequent of which states a require- 
ment of rationality governing the doxastic attitudes 
whose contents are so related: “N” stands for the govern- 
ing norm, and “a” and “p” are (possibly distinct) doxastic 
attitudes borne toward the propositions A;. On the basis of 
this blueprint, we can generate a number of distinct bridge 
principles by varying the following parameters: 


* Doxastic attitude: Does the principle govern full beliefs 
or credences (i.e., degrees of belief)? 

* Constraint on antecedent of the conditional: How (if 
at all) is the antecedent to be constrained? Does the 
norm cover all logical consequences? Or perhaps only 
those that are known, believed, obvious, and so on? 

* Deontic operator: Which deontic modal operator 
features in the requirement expressed in the conse- 
quent? Is it a claim about what the agent ought, has 
permission, or has (defeasible) reason to “do”? 

* Scope of operator: Typically, the principle’s conse- 
quent itself takes the form of a conditional—call it the 
embedded conditional. Ought the deontic operator 
O take wide scope with respect to it (ie., O(A = B))? 
Ought it to take narrow scope (A—O(B))? Should it 
attach to both the consequent and the antecedent 
(O(A) + O(B))? Or should we think of the normative 
claim as involving a primitive, undecomposable con- 
ditional operator (O(B | A))? 

* Polarity: What is the polarity of the normative claim? 
Is it a positive demand that the agent have a particu- 
lar attitude? Or is it a negative demand that the agent 
not have a particular attitude? 

* Synchronic—diachronic: Are the principles synchronic, 
instructing us how the agent’s doxastic state ought to 
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be at any given moment in time? Or are they require- 
ments about how one’s beliefs should evolve over 
time? 


A few examples may be useful. 


1. If Ai,..., A, EC, then S has reason to believe C if § 
believes each of the Ai. 

2. If Ai,..., An = C, then S’s credences ought to be such 
that® cr(C) > dy <<, Cr(A;) — (n—- 1). 

3. If Sknows that Ai,...,A,-C, then S ought not simul- 
taneously disbelieve C and believe each of the Aj. 

4. If S believes that Ai,...,A, FC, then, if S is permitted 
to believe each of the Aj, S is permitted to believe C. 


Principles 1 and 2 have unconstrained antecedents. In 
the first, the normative claim in the consequent features 
the reason-operator, which takes narrow scope with 
respect to the consequent of the embedded conditional 
and governs full belief. The second principle is famil- 
iar from probability logic (Adams, 1998) and has been 
advanced as a bridge principle by Field (2009, 2015). 
It governs credences, and ought effectively takes wide 
scope in the consequent. Principle 3 differs in that it 
is restricted to known consequences and of negative 
polarity. It too employs the ought-operator, which takes 
wide scope over the embedded conditional, although it 
governs full belief. Principle 4 is relativized to a nonfac- 
tive attitude. It features the permission-operator, which 
attaches both to the antecedent and to the consequent of 
the embedded conditional and also governs full beliefs. 

For simplicity, 1am assuming that all principles are to 
be understood as synchronic principles. That is, princi- 
ple 1 is to be read as 


(Synchronic) If S recognizes (at ft) that Ai,..., An FEC, 
then, if S believes each of the A; at t, S has reason to 
believe C at t 


as opposed to 


(Diachronic) If Ai,..., A,EC, then, if S believes each 
of the A; at t and t slightly precedes t', S has reason to 
believe C at t' (and from t’ onward). 


If there is an interesting connection between logic 
and rationality, there should be a viable bridge princi- 
ple that articulates it. But what counts as a viable bridge 
principle? We can distinguish two approaches. On one 
approach (Field, 2009, 2015; MacFarlane, 2004; Milne, 
2009; Steinberger, 2017b), we lay down various criteria 
of adequacy for our bridge principles. The principles that 
perform sufficiently well against these criteria are our con- 
tenders. The second approach—which I consider in sec- 
tion 6—proceeds from more fundamental epistemological 
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principles and seeks to derive a bridge principle from them 
(assuming such a principle is to be had). 


5. What Makes for a Viable Bridge Principle? 


Let us begin by considering the former approach. The ade- 
quacy criteria appealed to in the literature derive in part 
from Harman’s objections to potential bridge principles 
and in part from MacFarlane (2004). We can summarize 
them as follows: 


Belief Revision: Suppose I believe both A and A — B (as well 
as accepting modus ponens). The mere fact that I have 
these beliefs and that I recognize them to jointly entail 
B does not normatively compel any particular attitude 
toward B on my part. In particular, it is not generally 
the case that I ought to, or have permission to, believe 
B. After all, B may be at odds with my evidence, and so it 
would be unreasonable of me to follow modus ponens 
slavishly by, as it were, “adding B to my belief box.” 
The rational course of “action,” rather, when B is unten- 
able, is for me to relinquish my belief in at least one of 
my antecedent beliefs A and AB on account of their 
unpalatable implications. Belief revision seems to tell 
against narrow-scope principles, at least against strict 
versions of such principles involving ought and permis- 
sion. Similarly, because of the reflexivity of the conse- 
quence relation, these principles would imply that one 
ought to (or is permitted to) believe anything one in 
fact believes, which clearly seems problematic. Pinder 
(2017) defends a reasons-based narrow-scope principle 
(namely, principle 1 above) against such objections.’ 

Excessive Demands: Principles whose antecedents are 
unrestricted pose excessive demands on agents whose 
resources of time, computational power, stamina, and 
so on are limited. Take an ought-based narrow-scope 
principle according to which one’s beliefs ought to 
be closed under logical consequence. Anyone who 
believes the axioms of Peano-Dedekind arithmetic 
ought to believe every last one of its theorems, even if 
a theorem’s shortest proof has more steps than there 
are particles in the universe. But if the logical ought 
implies can (in the sense of what agents even remotely 
like us can do), such principles must be rejected.’ 

Clutter Avoidance: A related worry is this. Any of the 
propositions I believe entails an infinite number of 
propositions that are of no significance to me what- 
soever. Not only do I not care about, say, the disjunc- 
tion “Vienna is the capital of Austria or pigs can fly” 
entailed by my belief that Vienna is the capital of 
Austria, but it would be positively irrational for me 
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to squander my meager cognitive resources on infer- 
ring trivial implications of my beliefs that are of no 
relevance to my goals. 


Epistemic Paradoxes: Some maintain that there are vari- 
ous types of epistemic situations in which it is arguably 
not merely excusable for an agent to have logically 
incoherent beliefs but where such incoherence is per- 
missible or even rationally mandated. The Preface Par- 
adox arguably is a case in point (see Makinson, 1965). 
Here is a summary: suppose I am the author of a non- 
trivial nonfiction book. Having scrupulously checked 
the evidence for every one of my claims Aj, ..., An, I 
am highly confident (and believe) in each of them. 
I am also highly confident that at least one of my 
claims is fallacious. After all, Iam fallible. Call the prop- 
osition that =(A;a...AA;,) the “preface proposition” 
(P). Clearly, my beliefs are inconsistent. Moreover, 
given reasonable assumptions, in believing P, I ought 
to disbelieve a straightforward logical consequence of 
my beliefs, namely, their conjunction (Ain... AA,). 
The Preface Paradox presumably poses problems for 
all principles involving full belief (although credence- 
based principles may get around it). 


The Strictness Test: At least when it comes to ordinary, 
readily recognizable logical implications leading to 
conclusions that the agent has reason to consider, 
the logical obligation should be strict: there is some- 
thing amiss about an agent who endorses the prem- 
ises and yet disbelieves the conclusion on account of 
stronger countervailing reasons (MacFarlane, 2004, 
p. 12).'' The Strictness Test prima facie represents a 
strike against principles featuring the reason-operator, 
which countenances cases in which an agent believes 
the premises but disbelieves the conclusion on account 
of sufficient independent reasons for doing so. 


The Priority Question: The attitudinal variants have a 
distinctive advantage when it comes to dealing with 
Excessive Demands worries. But relativizing one’s log- 
ical obligations to, for example, one’s believed or rec- 
ognized logical consequences invites problems of its 
own, according to MacFarlane (2004): “We seek logi- 
cal knowledge so that we will know how we ought to 
revise our beliefs: not just how we will be obligated 
to revise them when we acquire this logical knowl- 
edge, but how we are obligated to revise them even 
now, in our state of ignorance” (p. 12). 

Logical Obtuseness: Suppose someone professes to 
believe A and B but refuses to take a stand on (nei- 
ther believes nor disbelieves) the conjunction A a B. 
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Intuitively, such a person is liable to criticism. 
Whereas the weaker (positive) reason-based princi- 
ples fail to live up to the Strictness Test, they do not 
commit the sin of logical obtuseness since one at 
least has reason to believe, or not disbelieve, AaB. 
Not so for principles with negative polarity. So long 
as the agent does not actively disbelieve Aa B, our 
negative bridge principles find no fault with cases 
like these. If this intuition carries any weight, nega- 
tive principles may prove to be ultimately too weak 
(at least on their own). 


There is little consensus over which bridge principle 
fares best in light of these desiderata (for an overview, 
see Steinberger, 2017b). One important sticking point is 
how to deal with the Preface Paradox and similar cases. 
MacFarlane (2004) argues that we must simply resign 
ourselves to the existence of an irresolvable normative 
conflict between the demands placed on us by logic and 
other epistemic norms. Field (2009) endorses principle 2, 
which constrains credences. The principle allows for 
one’s credences in a consequence jointly entailed by all 
the propositions (e.g., their conjunction) to be very low, 
despite one’s having high credence in each of the indi- 
vidual propositions. The principle is compatible with a 
broadly Bayesian view of credences, according to which 
a rational agent’s credence function is (or is extendible 
to) a probability function. Another important issue is 
the question of the permissible level of idealization of 
a requirement of rationality. If our aim is to formulate 
principles of ideal rationality, Clutter Avoidance and 
Excessive Demands may matter less. But how, then, do 
these principles relate to ordinary agents like you and 
me (Harman, 1986)? 

Iargue (Steinberger, 2017a, 2019) that the approach as a 
whole is marred by our failure to distinguish three impor- 
tantly different ways in which logic might be normative: 


Directives provide first-personal guidance in the process 
of doxastic deliberation. 

Evaluations serve as objective third-personal standards 
or ideals for classifying acts or states into correct and 
incorrect ones. 

Appraisals serve as the basis for our (equally third-personal) 
criticisms of our epistemic peers and so underwrite our 
attributions of praise and blame. 


To illustrate, consider the following act-utilitarian principle: 


AU You ought to act in such a way as to maximize net 
happiness. 
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The principle might be an apt evaluative norm in that it 
serves as a metric for what is to count as a right action. 
Yet the norm is often of little help to an agent trying to 
figure out what to do. Typically, it will not be transpar- 
ent to her which of the actions available to her maxi- 
mize happiness. The norm offers the agent little by way 
of guidance and therefore is not fit to play the role of 
a directive.'? Similarly, if our agent violates the evalua- 
tive utilitarian norm, she may nevertheless not be liable 
to criticism. Despite having violated the norm, she may 
have acted reasonably in light of how the situation pre- 
sented itself to her. Conversely, she might have acted 
recklessly and yet, out of sheer luck, complied with the 
norm. In both cases, our appraisals and our evaluations 
come apart, and this is largely because our appraisals 
are, while our evaluations need not be, sensitive to the 
agent’s perspective. 

I argue (Steinberger, 2019) that this tripartite distinc- 
tion reveals that the actors in this debate talk past one 
another on account of conceiving of the normative role 
of logic differently. Another advantage of the proposed 
analysis is this. The adequacy criteria are in tension 
with one another, leading participants in the debate 
to make unprincipled choices about which desiderata 
to discount in order to make a case for their favored 
principle(s). The threefold distinction has the virtue of 
explaining why our adequacy criteria are inconsistent 
and points us toward a philosophically well-motivated 
resolution. The criteria are inconsistent because they are 
motivated by the aforementioned incompatible con- 
ceptions of the normative role of logic. Once we distin- 
guish between directives, evaluations, and appraisals, 
we find that each normative role is naturally associated 
with its own set of adequacy criteria, which form con- 
sistent subsets of our desiderata. Hence, for each nor- 
mative role, there is a separate, well-defined, and—it is 
hoped—more tractable question as to whether logic is 
normative in that sense. 

Even if my analysis is correct, though, it remains to be 
seen which (if any) of these normative roles could give 
rise to principles of rationality. 


6. Logic and Epistemic Utility Theory 


Let us now turn to the second approach. On this view, 
norms of epistemic rationality are to be derived from 
fundamental epistemic aims or values. Several candi- 
dates could be in the running: truth, knowledge, under- 
standing, and so on. Also, we may ask whether we 
should be monists or pluralists about epistemic value. 
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Finally, given the value(s) at the center of our episte- 
mology, what is its normative structure and how does 
it give rise to a theory of epistemic rationality? Rather 
than attempting a necessarily superficial survey of the 
options here, I propose to study a concrete proposal: 
epistemic utility theory (EUT). The prevalent version of 
EUT elects truth as its sole and fundamental epistemic 
value. What is more, the theory’s value-theoretic super- 
structure is broadly consequentialist: any principle of 
epistemic rationality must earn its keep by promoting 
the aim of truth. 

William James’s famous slogan “Believe truth! Shun 
error!” serves as a good starting point.’ While belief 
aims at the truth, our attempts at achieving it are subject 
to this double imperative. If my sole aim was the maxi- 
mization of true belief without any concern for error, I 
might as well believe any proposition whatsoever. Con- 
versely, if I was so cautious as to wish to avoid error at 
all costs, I would be best off suspending belief across the 
board. Our challenge, then, is to strike the optimal bal- 
ance between maximizing true beliefs and avoiding false 
ones. EUT proposes to bring the tools of decision theory 
to bear on the problem."* 

Let us consider the case of full belief. Suppose for sim- 
plicity that you are entertaining propositions belonging 
to a finite set #. For each proposition in Y, the agent 
may believe (B), disbelieve (D), or suspend belief (S). 
Formally, we can represent the agent’s various possible 
“choices” over FY as belief functions b: A = {B, S, D}. Our 
belief functions are to be assessed based on their accu- 
racy. Our standard of accuracy is the truth-value of the 
proposition at a possible world, where a world is simply 
represented by means of a consistent valuation function 
w: P = {t, f}. 

In analogy with decision theory, we can conceive 
of each choice of a doxastic attitude relative to a world 
as producing a certain epistemic utility, which is repre- 
sented by a numerical value. Epistemic utility can be 
represented by a further function that associates such a 
value to any attitude—truth-value pair: 


eu: {B,D,S} x {t, f} > (—0e, 0). 


That is, for any proposition A © Y, eu returns a score as a 
function of one’s attitude toward it and the proposition’s 
truth-value at the world. R is the score for getting it 
right, -W is the penalty one incurs for getting it wrong, 
and suspending yields a neutral score. Hence, 


eu(B, t) = eu(D,f) = R, 
eu(S,t) = eu(S,f) = 0, 
eu(B,f) = eu(D,t) = -W. 


210 


How are we to conceive of the relative values of R and 
W? There are three options: 


* The epistemic radical values true belief to a higher 
degree than she disvalues false belief: R > W. 

* The epistemic centrist values both to the same degree: 
R=W. 

* The epistemic conservative disvalues false belief to a 
higher degree than she values true belief: W > R. 


James believed one’s stance toward the question to be a 
matter of intellectual temperament. There is, however, 
a prima facie case to be made for conservativeness. Sup- 
pose I flip a fair coin. Let p be the proposition that the 
coin lands heads. What attitude is it rational for me to 
adopt with respect to p? It is clear that in the absence 
of further information I should suspend. However, con- 
sider the following decision matrix: 


Pp “p BB BD SS BS 
t f R-W 2R 0) R+0 
f t -W+R -2W 0 -W+0 


(The remaining cases—DD, DB, SB, DS, SD—are strictly 
analogous and so can be omitted.) For the epistemic 
radical, the optimal choice is to believe p and disbelieve 
=p, or vice versa. But blindly going out on a limb in this 
way seems reckless. The centrist is no better off. She is 
indifferent between believing (or disbelieving) both p 
and its contradictory, believing p and disbelieving =p 
(or vice versa), and suspending.'® Conservatism, then, is 
our best option: the disvalue of believing falsely outstrips 
the value of believing truly. Since having contradictory 
beliefs always adds a net negative to one’s score (R- W), 
one would be irrational to do so. 

Having thus fixed the relative value of our rewards 
and penalties, we can determine the overall epistemic 
utility of a belief function at a world. Epistemic utility is 
generally assumed to be additive:'° the overall epistemic 
utility of b, EU(D), is 

EU(b)= > eu(b(A), w(A)). 

AeP 
In general, of course, we are interested in the actual 
world and will do our best to choose the belief function 
with the greatest actual epistemic utility. However, we 
generally do not know which of our beliefs are accurate. 
The score of a belief function then hinges on facts about 
the world of which we are uncertain. What we can be 
certain of, though, is that a belief function that has less 
epistemic utility than another, however the world turns 
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out to be, can be eliminated. It would be plainly irratio- 
nal to adopt it. Not only is such a belief function bound 
to be suboptimal, but I might have appreciated its sub- 
optimality even in the absence of any knowledge about 
the actual world. Decision theory captures this form of 
irrationality in terms of the notion of dominance: 


* A belief function b is strictly dominated by a belief 
function b' iff EU(b’, w) > EU(b, w) for all worlds w. 

* A belief function b is weakly dominated by a belief 
function b' iff EU(b', w) = EU(b, w) for all worlds w and 
there exists a world w’ such that EU(b’, w’) > EU(b, w’). 


Two brief remarks. First, one option’s being dominated 
by another makes choosing it irrational only if the domi- 
nating option is not itself dominated. Consider this. A 
genie grants me a (single) cash reward: for any number 
n I care to specify, I receive exactly $n. But of course, any 
choice of n is bound to be dominated. That is not to say, 
though, that there is no rational response (see Pettigrew, 
2016a). Second, note that a belief function b’s being dom- 
inated by another (undominated) function b' only shows 
that it would be irrational to opt for b. It says nothing 
about which belief function one should adopt (in particu- 
lar, it does not in general recommend adopting b’). 

Following Easwaran (2016), we can say that a belief func- 
tion b is strongly coherent just in case it is not even weakly 
dominated and that b is weakly coherent just in case it is not 
strongly dominated. The following rationality requirement 
thus falls right out of EUT’s central commitments: 


Strong Coherence: One ought to have strongly coherent 
beliefs. 


Central for our purposes is the question of how logical 
principles of rationality relate to Strong Coherence and, 
more generally, whether they can be justified in the con- 
text of EUT. 

First, let us observe that Strong Coherence requires 
that one believe all logical truths: 


Logical Truth: If S’s belief function is strongly coherent, 
then S believes all logical truths (in 7). 


The following table illustrates why: 


B S D 
A R -W 
aA -wW 0 R 


Suppose A is a logical truth. It follows that the second 
line in bold can be ignored because there is no logically 
possible world at which A is false. The score of one’s 
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belief function is thus determined on the basis of the 
first row of the table alone. Thus, any belief function b 
that suspends belief in, or disbelieves, a tautology will 
have less epistemic utility at every possible world than a 
belief function b’ that agrees with b on all propositions 
aside from A, which it believes. 

Next let us consider the following weak single- 
premise bridge principle. The principle is wide in scope 
and of negative polarity. Moreover, it is restricted to 7. 


Single-Premise Closure: For all A, CE, if AFC, then S$ 
ought not both believe A and disbelieve C. 


It turns out that that Strong Coherence entails Single- 
Premise Closure. Suppose belief function b violates 
Single-Premise Closure. That is, assume that A, CEP 
and b believes A and disbelieves C. Then bis strictly dom- 
inated by the belief function b’, which suspends on both 
propositions, as the following table illustrates (where, as 
above, the second line in bold, wz, can be eliminated): 


AFC A Cc BD SS 
Wi t t R-W 0) 
W3 f t -2W 0 
W4 f f -W+R 0 


The question is whether analogous results are to be had 
for the following two familiar requirements: 


Multiple-Premise Closure: For all {Ai,..., An, C} CF, if 
Ai,..., An FC, then S ought not both believe all of 
the A; and disbelieve C. 

Consistency: If {Ai, ... , An} is a logically inconsistent set, 
then S ought not believe each member of the set. 


At the risk of ruining the suspense: Strong Coherence 
entails neither Multiple-Premise Closure nor Consis- 
tency. To see this, a small detour is necessary. First, let us 
introduce the notion of expected epistemic utility. The 
expected epistemic utility of a belief function b is the 
probability-weighted sum of its epistemic utilities across 
all possible worlds: 

EEU, (b) = >, P(w)EU(b,w). 

weW 
The probability function P can be thought of as the sub- 
ject’s evidence or at least as partially determined by her 
evidence. The probability function might be interpreted 
as a credence function, as an evidential probability func- 
tion, or as representing objective chances, depending on 
one’s conception of evidence. 
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It can be shown that it is a sufficient condition for a 
belief function to be strongly coherent that there be a 
regular probability function relative to which it maxi- 
mizes expected utility.’ What is more, it can be shown 
that a belief function b has maximal expected utility just 
in case there exists a probability function P such that for 
any proposition A,"* 


» b(A) =B iff 1=P(Ay= —Y_, 
R+W 
» p(Ay=Siff W 2payz_F 
R+W R+W 
- D(A) =D iff _R_ =P(A)=0. 


R+W 


It is noteworthy that if P is interpreted as the agent’s 
credence function, this result yields a version of the so- 
called Lockean thesis, according to which fully (ratio- 
nally) believing is (in a sense to be made precise) having 
credence in excess of a threshold. The threshold is set by 
the appropriate ratio between R and W.”” 

The upshot is that belief functions can maximize 
expected utility (and hence be strongly coherent) while 
satisfying neither Multiple-Premise Closure nor Consis- 
tency. The Preface Paradox can again serve as an exam- 
ple. Suppose our author entertains the propositions 
Ai,..., An, Which compose the body of her book, and 
that she has rational uniform credences with respect to 
these propositions. That is, her credence function cr is 
(or is extendible to) a probability function, and she has 
the same high credence, say, cr(Ai) = .9, with respect to 


each of the propositions. Assuming that .9 = Raw the 


+W’ 
belief function b that believes all of the A; maximizes 
expected utility with respect to cr and so is strongly 
coherent. However, provided that n is sufficiently large 
and that the author entertains Ain ...”A, and its nega- 


tion, we get cr(Aia...AAn) s , and so B, if it is 


R+W 

to maximize expected utility, disbelieves Ain... AAn 
(and believes its negation). The belief function b is then 
strongly coherent and yet violates both Multiple-Premise 
Closure and Consistency. We can resist this conclusion 
only if we ensure that W = (n- 1)R. Here n (we are assum- 
ing n> 2) is the number of propositions in question. EUT 
requires consistency and full closure just in case the dis- 
value of error exceeds the value of true belief by a factor 
of n-1 (see Easwaran & Fitelson, 2015). 

What are we to make of this? Here is an observa- 
tion. Strong Coherence and Consistency are structurally 
very similar. Violations are, in both cases, a priori know- 
able indications that one’s beliefs are less accurate than 
alternative ones. The difference comes down to a subtle 


212 


quantifier shift: if b is an incoherent belief function, then 
there exists a belief function b’ that has at least equal 
epistemic utility across all worlds and outperforms it in 
some. That is, 


3b'(Ww EU(b’, w) = EU(b, w) a 
Jw’ EU(b',w') > EU(b, w’)). 


By contrast, if b is an inconsistent belief function, then, 
for every world w, there exists a belief function b’ such 
that EU(b', w) > EU(b, w), that is, 


Wwab' EU(b', w) > EU(b, w). 


According to EUT, the former violation spells irratio- 
nality, while the latter does not. But why should this 
minor difference be of such moment? If our aim really 
is the truth, then why should we content ourselves with 
maximizing epistemic utility in cases where, because our 
beliefs are inconsistent, some of them are bound to be 
false? Not only is maximizing expected epistemic utility 
no necessary condition for perfect accuracy, but it may 
sometimes positively discourage one from holding a true 
belief, because doing so would come with too high an 
opportunity cost: it would lower the expected epistemic 
utility of the belief function as a whole. Cases of such 
counterintuitive epistemic trade-offs have been the sub- 
ject of much discussion.” 

Many of these criticisms target the epistemic value 
theory undergirding EUT: that epistemic rationality is 
teleologically structured and that principles of rational- 
ity are justified only inasmuch as they further expected 
epistemic value (see Littlejohn, 2016). 

Others question the assumed monism about epistemic 
value: is truth really the sole source of epistemic value? 
And even fellow monists may disagree about the nature 
of the fundamental aim. Why should truth be the North 
Star of epistemology as opposed to, for instance, knowl- 
edge?”' Accidental true belief is not enough, they say. We 
want to get it right for the right reason. Finally, Leitgeb 
(2013, 2017) has argued for the possibility of providing a 
harmonious account of doxastic rationality according to 
which a probabilist approach to rational credence is com- 
patible with a logical approach to rational full belief that 
incorporates full consistency and closure requirements. 


Notes 


1. Epistemic (or theoretical) rationality, here, is contrasted with 
practical rationality. Roughly, the former is concerned with what 
we ought to believe, and the latter is concerned with what we 
ought to do. 


2. It might be thought that the two forms of coherence really 
are the same. Some, most notably Niko Kolodny (2007, 2008), 
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maintain that there is no separate demand that beliefs should 
cohere. To the extent that beliefs should indeed cohere, this 
is because beliefs that enjoy evidential support cannot fail but 
cohere in the right way. I will set Kolodny’s challenge aside 
for the purposes of this discussion. Conversely, it might be 
thought that one’s evidence is constituted only by one’s 
beliefs. If so, a belief is evidentially supported precisely when it 
coheres with the entire body of relevant beliefs. 


3. It is a matter of some controversy whether requirements of 
rationality have genuine normative force. While I allow myself 
to use formulations such as “norms of epistemic rationality,” 
I do not mean to prejudge the issue. 


4. For ease of exposition, I do not include function symbols, 
nor am | including identity among the logical constants. 


5. In the case of atomic formulas of the form P’(t), we simply 
identify—for uniformity’s sake—the “1-tuple” (d) with d. Hence, 
the extensions of one-place predicates continue to be sets of ele- 
ments of D and not of 1-tuples of such objects. 


6. For a much more detailed exposition, the interested reader 
may consult the excellent collaborative open-source textbook 
by Magnus and Button (2019) or Enderton (2001). 


7. Henceforth, I use “logic” to mean FOL unless I explicitly 
indicate otherwise. Similarly, “F” designates the consequence 
relation of FOL. 


8. Call u(A) the uncertainty in A, defined as u(A) = 1-cr(A). 
We can then give the following tidier reformulation of the 
inequality: u(C) < dy <;<, u(A)). 


9. In so doing, Pinder (2017) takes issue with an argument by 
Steinberger (2016) to the effect that appeals to the normativity 
of logic do not support a case for paraconsistent revisions of 
our logics (logics that reject the principle of explosion, accord- 
ing to which from an inconsistent set of propositions any 
proposition whatsoever logically follows). 


10. It is for worries like these that Isaac Levi (2002) spoke 
of doxastic commitments, which do not demand the agent 
to believe but rather amount to something like a promise to 
believe. For similar issues in epistemic and doxastic logic, see 
chapter 5.1 by van Ditmarsch (this handbook). 


11. MacFarlane takes inspiration from Broome (2000, p. 85). 


12. One need not maintain that directives must always have 
transparent application conditions (see Srinivasan, 2015). 


13. See Pettigrew (2016c) for a formal analysis of James’s 
insights. For an alternative analysis, see Raidl and Spohn (2019). 


14. The following exposition is inspired by Easwaran (2016) 
and Pettigrew (2017). 


15. See also Easwaran and Fitelson (2015) as to why W sR leads 
to counterintuitive consequences. 


16. This assumption is not essential (see, e.g., Dorst, 2019). 
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17. A regular probability function, in the present context, is 
one that assigns every world a nonzero probability. 


18. See Easwaran (2016) for details. 


19. Dorst (2019) argues for a more sophisticated variable- 
threshold Lockean account, which he takes to provide a meta- 
physical reduction of full belief to credence. Easwaran (2016), 
by contrast, argues in favor of the primacy of full belief. 


20. See Berker (2013), Carr (2017), Greaves (2013), and Little- 
john (2012, 2015) for criticisms along these lines as they per- 
tain to EUT and epistemic consequentialism more broadly. See 
Pettigrew (2016b) for a response. 


21. The go-to reference for much of the contemporary debate 
is Williamson (2000). 
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3.2 Natural Logic 


David P. O’Brien 


Summary 


Declarative memory requires a predicate-argument struc- 
ture to record propositions, and this is provided by a 
natural logic. A natural logic requires a representational 
format, some inference procedures, and a program for 
implementing the procedures and must attend to prag- 
matic influences. I show how inferences from a natural 
logic are integrated in reasoning with inferences from 
other practical processes, although a reasoner is not 
aware of the various origins of their inferences. I argue 
that other theories that eschew any role for a natural 
logic are not adequate without being integrated with a 
natural logic. 


1. The Historical Shift in Attitudes about 
Rationality and Logic 


The pre-Socratic Greeks introduced the idea that humans 
are rational, and after Aristotle, the basic Western tradi- 
tion identified human rationality with logic—a perspec- 
tive that dominated the Western perspective until the 
end of the 19th century, with only an occasional chal- 
lenge by the likes of Bacon, Hume, or Nietzsche. Indeed, 
Boole (1854) treated logic as the study of human mental 
acts, so logic and human thinking were understood as 
more or less equivalent and part of the essence of human 
nature. The transition from the 19th to the 20th century, 
however, brought challenges to this view as psychology 
separated from philosophy after the founding of the 
new science late in the 19th century. 

Although the most obviously popular claim that 
humans are not basically logical is found in Freud (e.g., 
1914), work by logicians also encouraged the separation 
of logic from human thought. Philosophers like Frege 
and Russell, for example, argued for the separation of 
logic and epistemology from psychology (e.g., Gabbay 
& Woods, 2009; George, 1997; Passmore, 1994), and as 
professional logicians have focused on finding a founda- 
tion for mathematics in logic with their emphases on 


complex proofs of soundness and completeness, they 
were not producing something inviting to someone 
seeking a model for ordinary human reasoning. At the 
same time, the newly established field of psychology was 
eager to divorce itself from philosophy (Boring, 1929; 
Mandler, 2007), seeking psychological explanations that 
were not imported from philosophical logic. So psychol- 
ogy turned away from logical rationality to rewards, 
punishments, reinforcers, instincts, defense mechanisms, 
and the like, and philosophers turned away from psy- 
chology to construct mathematical systems of logic that 
provided little of obvious interest psychologically. Henle 
(1962) noted, however, that these movements of drift 
between logic and psychology were based not on evi- 
dence but on a shift in attitude. 

A shift in intellectual attitude is not an argument, and 
the fact that the assumption of human logical rationality 
had a two-and-a-half-millennia history should encour- 
age us to reconsider its abandonment, keeping in mind 
that many different approaches have been proposed by 
logicians, and the lack of fit of any of them to psychol- 
ogy is not an argument that human reasoning lacks a 
logic. The stoic logic of Chrysippus, for example, was 
quite different from Aristotle’s logic, and one might find 
things in Chrysippus that are psychologically interesting 
in ways that Aristotle’s logic is not. The logics one finds 
in a contemporary standard-logic textbook, and in the 
professional journals for logicians with their truth tables 
and elaborate proofs of consistency and completeness, 
were not constructed to be models for human thought. 
To discover what kind of logic should be natural for 
humans, we need to think about why humans need a 
logic, what that logic would need to include, and what it 
would be unlikely to include. 


2. Why We Should Expect a Natural Logic 
and What It Should Include 


I begin with the fact that humans have a declarative 
memory, and it is axiomatic that to record something in 


216 


declarative memory, a format must exist with which to 
record it (Braine & O’Brien, 1998; Fodor, 1981; O’Brien & 
Li, 2013). This format would need to keep track of enti- 
ties and the properties that attach to those entities, 
and people hold propositional attitudes about what is 
expressed in the propositions that contain the entities 
and their properties, so the format must keep track of 
what is true and what is false so that people can assert, 
deny, suppose, question, attack, defend, believe, and 
doubt their propositions. Such a format is tantamount 
to a predicate-argument structure, so the approach 
described here begins with the assumption that the 
mind should include a logical format with which to rec- 
ord propositions. This logical format should bear some 
interesting relationship to the kinds of standard logics 
one finds in logic textbooks, although by no means 
would we expect it to be identical to any textbook 
logic, and we have referred to our approach as a natu- 
ral logic or a mental logic as it is intended to provide 
an account of reasoning rather than to construct a formal 
system that is sound and complete. The task is different 
from that of a logician. Indeed, it begins by acknowl- 
edging that completeness of human logical reasoning 
has not been a consideration in human bioevolutionary 
development, although one would want their thinking 
to be sound in at least some meaningful way. The first 
task for a natural-logic theory is to describe the format, 
and once the format has been established, we need to 
address how inferences are drawn in the format, what 
sort of logical semantics is required, and how the natu- 
ral logic integrates with inferences that stem from prag- 
matic concerns. 

Standard-logic textbooks provide a language that 
includes symbols for connectives or operators (e.g., A, 
v, 7, J) and quantifiers (e.g., V, 4), and such symbols 
correspond loosely to ordinary words of daily language 
(e.g., and, or, not, if, all, some). They also provide a truth- 
functional semantics in the form of truth tables, with If 
p then q given the truth-functional assignment of being 
true unless p is true and q is false—a definition that 
makes true such propositions as I did not have a gun, but 
if I did, then I’m guilty as charged, which few people would 
want to utter. 

Propositions of this latter sort exemplify the para- 
doxes of material implication, and they provide a seri- 
ous problem for an attempt to argue that the truth table 
for if can provide a sensible model for how people reason 
with if-propositions. Our natural-logic approach thus 
looks elsewhere to find the meanings of logic operators. 
Compare the following two propositions: 
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All the children who went to the zoo were in the third 
grade. 


Some of the children who went to the zoo were in the 
third grade. 


Suppose we discover that Alvaro went to the zoo, and 
we ask whether we can infer that Alvaro was in the third 
grade. The proposition beginning with all provides a 
basis for the inference that he was in the third grade, but 
the proposition beginning with some does not. Indeed, 
someone who does not understand which sentence 
sanctions the inference and which does not could not 
be said to understand the meaning of the words all and 
some. This illustrates how the natural-logic approach 
understands the logical semantics for its vocabulary. The 
meanings of the terms are given by the inferences that 
they sanction. The tradition of understanding semantics 
in terms of the inferences they sanction follows Witt- 
genstein (1958), Gentzen (1935/1964), Harman (1982), 
and Block (1998), among others, and has become known 
as a “conceptual-role” or “inferential-role semantics.” 
This approach has been presented as a general theory 
for semantics across the lexicon, but our natural-logic 
approach makes no claims beyond the vocabulary of the 
natural logic, and for this part of the lexicon, a semantics 
based on the inferences that a term sanctions is straight- 
forward because the schemas and reasoning program 
of natural-logic theory provide a simple and succinct 
description of what the forms are for those inferences. 

I turn now to the question of the inference proce- 
dures, both because these procedures provide the logical 
semantics and, moreover, because an intelligent species 
needs inferential procedures that go beyond the infor- 
mation given (e.g., Bruner, 1997). Our hunter-gatherer 
ancestors would have benefited from an ability to make 
simple and immediate inferences. Imagine a gatherer in 
the forest who knows that the best locations for find- 
ing food at this time of the year are either next to the 
river or in the forest near the canyon. She encounters 
her cousin who has just returned from the riverbank and 
tells her that there is little to collect there this year as 
locusts have been there, so a simple and direct inference 
can be made that she is better served going to the forest 
near the canyon. Surely our ancestors needed to make 
such inferences to have survived and produced progeny, 
for it was their logical reasoning that kept them from 
needlessly visiting multiple sites in useless searches for 
food. This particular kind of inference can be captured 
by an inference schema for disjunctions, that is, alterna- 
tives, and the stoic logician Chrysippus attributed such 
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a schema to dogs following a trail of scent as well as to 
humans. (For discussions of why immediate inference 
procedures are not provided by the way professional 
logicians deal with truth conditions, see Braine, 1978; 
Dopp, 1962; Fitch, 1973; Gentzen, 1935/1964; Lakoff, 
1970.) 

Simply having some inference-making schemas would 
not be sufficient: a reasoner would need some sort of 
reasoning program that applies the schemas to produce 
lines of reasoning. Further, given that our thoughts usu- 
ally refer to practical concerns, an adequate theory of 
logical reasoning also would require an account of how 
the logical inferences are coordinated with inferences 
that refer to practical knowledge and practical interests. 
To exclude inferences that go beyond what is available 
on logic alone would be irrational on the part of investi- 
gators into rationality. An investigator of human reason- 
ing should not interpret inferences made in laboratory 
reasoning tasks that do not follow from logic alone as 
irrational. Such inferences may well be quite rational 
even when not inferable on logical grounds alone. 

This chapter presents the natural-logic theory of Braine 
and O’Brien (e.g., 1991, 1998) as a case study in how 
a natural logic can be developed. The presentation of 
the theory has three parts: the representational format 
(section 3), a set of reasoning schemas and a reasoning 
program that implements the schemas in constructing 
lines of reasoning (section 4), and a set of pragmatic 
principles that describe the interactions of the logical 
schemas with epistemic knowledge and practical goals 
(section 5). 


3. A Representational Format for a Natural Logic 


Standard-logic textbooks first introduce the sentential 
level, which includes atomic propositions and connec- 
tives for negation, conjunction, disjunction, condition- 
als, and so on. A predicate level adds quantifiers and a 
predicate—argument structure, which can make it appear 
that reasoning at the predicate level is more difficult, but 
natural logic need have no such expectation. 
Constructing a natural logic at the sentential level 
requires fewer decisions than does the predicate-logic 
level, providing only connectives but not quantifiers and 
the structure that quantifiers entail. The connectives in 
a natural logic are not identical to those in a standard 
logic. For example, “If p then q” is not equivalent to the 
material conditional of pDq, which as discussed earlier 
makes “If p then q” true whenever p is false. So the natu- 
ral logic presented in Braine and O’Brien (1998) eschews 
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the symbols one finds in logic textbooks and represents 
sentential connectives with italicized English words: if 
for conditionals, and for conjunction, or for disjunction, 
and not for negation, although we understand that these 
words have additional pragmatic interpretations that go 
beyond their purely logical meanings, and in ordinary 
speech, words can convey meanings that differ from 
their ordinary surface meanings. A mother who tells a 
child, “Put your hand in the cookie jar and I’ll send you 
to your room,” is using and where one formally would 
expect the word if, and we would not think the propo- 
sition requires that “The hand is put in the cookie jar” 
and “The child will be sent to their room” are both true 
in order for the assertion to be true. Even though on 
the surface, the assertion uses a word usually reserved 
for conjunction, the meaning intended clearly is condi- 
tional. Meaning thus is not found in the surface struc- 
tures of sentences but in the propositions that are held 
in a deeper language of thought. Linguistic input thus is 
translated into a language of thought from the language 
of speech, and then inferences made in the language of 
thought are translated back into the language of ordi- 
nary speech (see Braine & O’Brien, 1998; O’Brien & Li, 
2013). 

Constructing the representational format at the 
predicate-logic level requires that some decisions be 
made about the internal structure of propositions. Con- 
sider a universally quantified disjunction in a standard- 
logic textbook: 


(Wx)(Px v Qx), “For all x, x satisfies P or x satisfies Q,” 


where the universal quantifier is placed outside the prop- 
osition that is being quantified, and the quantifier has 
scope over all instances of the bound variables within 
the parentheses, whatever the content predicates are to 
which the variables are attached. 

A corresponding natural logic representation is: 


S,[All_X] or S2[PRO-AIl X], 
“All the X’s satisfy S; or they satisfy 52,” 


where the scope of quantification is inside the expression 
and is specific to particular content domains. Quantifica- 
tional scope in this example is marked by a pro-notation, 
referring to the pronoun in the second clause that is an 
anaphoric reflex to the quantifier in the first clause. This 
choice about how to represent quantification reflects the 
use of pronouns that is typical in natural languages as a 
way to convey quantificational scope. Ioup (1975) sur- 
veyed 14 languages and found no evidence for the type of 
outside scope that is typical in standard-logic textbooks; 
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instead, quantifiers tend to be lexicalized inside noun 
phrases, as they are in the natural-logic representations, 
and quantification is bound to the specific content, as it 
tends to be in natural languages. 

One needs to distinguish between alternative inter- 
pretations of the surface grammar of sentences like “All 
the boys are carrying a box,” where one interpretation is 
that there is a single large box that is being carried by a 
group of boys, and a different interpretation is that each 
boy is carrying a different box from the other boys. The 
first interpretation can be captured by a representation 
of this sort: 


CARRY BOX (BOYS), 


and the second interpretation can be captured by one 
of this sort: 


CARRY BOX (All Boys). 


Such sorts of collective sets are not at all unusual. Braine 
and O’Brien (1998) provided the example 


“The prosecutors convinced the 12 members of the jury of 
the defendant’s guilt,” 


CONVINCED (PROSECUTORS) (AII JURORS), 


where the collective group of prosecutors convinced 
each individual jury member. 


4. Inference Schemas and a Reasoning Program 


Natural-logic theories propose that people construct 
lines of reasoning governed by inference schemas (e.g., 
Braine & O’Brien, 1991, 1998; Fitch, 1973; Gentzen, 
1935/1964; Rips, 1994), and Braine and O’Brien empha- 
sized the need to seek empirical evidence for guidance 
about which procedures used to construct inferences 
in lines of reasoning should be included. For example, 
Braine, Reiser, and Rumain (1984) found that college stu- 
dents tend not to accept “p or q” as following from p 
alone, even though it is a valid inference in standard 
logic and is included in the natural logic of Rips (1994); 
it was excluded on empirical grounds by Braine and 
O’Brien (1998). 

Braine and O’Brien (1998, tables 6.1 and 6.2) pre- 
sented a set of 14 schemas at the sentential level together 
with a reasoning program that applies the schemas to 
construct lines of reasoning. I describe these in some 
detail here to provide a sense of what is involved. Seven 
schemas are called “core schemas,” and they are applied 
without restriction by the reasoning program whenever 
their conditions for application are met. For example 
(presented here in simplified form), when one holds “p 
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or q” and not-p together in working memory, one infers 
q, as did our gatherer described earlier who decided to go 
to the area of the canyon to procure food. Another core 
schema is modus ponens, which holds that when one 
considers “If p then q” and p jointly, one infers q. 

Two schemas are called “feeder schemas,” and their 
application is restricted so that they are applied only 
when their output can feed another inference. One of 
these infers “p and q” when both p and gq are in working 
memory, and the other infers p alone (or infers q alone) 
from “p and q.” The restriction is required because if left 
unrestricted, these feeder schemas could produce infi- 
nite loops. Also, people do not report their output when 
asked to write down everything that follows from some 
premises, yet such schemas seem to exist because people 
apply, for example, modus ponens to “If p and q then r” 
when they have both p and q separately. 

The final four schemas include two that pertain to 
incompatibility and two that pertain to supposition. 
Supposition occurs when a proposition is treated as true 
for the purposes of reasoning with the supposed propo- 
sition, even when one does not know whether or not it 
is true. One of the supposition schemas combines with 
an incompatibility schema so that the supposition can 
be falsified as in a reductio ad absurdum argument. The 
other is a schema for conditional proof, which is used to 
infer propositions of the form “If p then q” or to evalu- 
ate whether “If p then q” follows from a set of prem- 
ises. To infer a proposition of the form “If p then q,” 
first suppose p. When a set of premises and background 
assumptions together with the supposition of p lead to 
the inference of q, this schema provides the inference of 
“Tf p then q.” 

The schemas are applied by a reasoning routine, which 
has two parts: a direct-reasoning routine (the DRR) that 
we claim is universally available and applied automati- 
cally, and some reasoning strategies that go beyond the 
DRR and are widely available among college students. 
The schema for conditional proof has a procedure in the 
DRR that applies when a line of reasoning is constructed 
to evaluate “If p then q”: the proposition p is added to 
the premises as a suppositional premise, and reasoning 
proceeds toward g. When q is derived, “If p then q” is 
evaluated as true. When not-q is derived, “If p then q” 
is evaluated as false. The latter evaluation would not be 
made in standard logic with its material conditional— 
because p might be false, even when supposed as true, 
although both young children and adults seem to use this 
procedure (Braine & O’Brien, 1998, chapters 7 and 17). 

Note that the combination of the schemas and the 
reasoning program enables precise predictions about 
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what inferences people make, and some of these infer- 
ences differ from what would be predicted if natural rea- 
soning were following standard textbook logic, like the 
evaluations of when conditional propositions are false 
just described. The combination of schemas and reason- 
ing program also helps us understand the consistency of 
the schema for conditional proof and the modus ponens 
schema, which together provide the inferential seman- 
tics for if, Given that “If p then q” is derived by supposing 
p and then showing that this entails g, the introduction 
of p as a premise when one knows “If p then q” allows 
the conditional nature of q to be discharged so that q 
alone can be asserted. That is, modus ponens releases q 
from requiring the marking of p as suppositional when 
one knows that p is true, because the schema for condi- 
tional proof ensures that q is true when p is. 

Reasoning under a supposition requires a constraint 
(see Braine & O’Brien, 1991). A sound line of reasoning 
can proceed only from true premises, and when a line 
of reasoning is done under a supposition, one needs 
to be sure that the propositions introduced under the 
supposition would still be true given the supposition. 
When it is not clear whether a proposition should be 
excluded under a counterfactual supposition, logic per 
se provides no clear way to resolve such cases, and inter- 
locutors would need to negotiate about whether exclu- 
sion is required. Note, by the way, that the natural logic 
is able to deal with counterfactual conditionals without 
recourse to some special semantics that requires refer- 
ence to a possible world that is most similar to the actual 
world. After all, what would the possible world be that 
is most close to the actual world when we consider the 
counterfactual “Philadelphia is north of New York City”? 
One would have to decide to move New York southward 
or to move Philadelphia northward, with all of the messy 
repercussions that such a decision would have. 

For a detailed presentation of the complete set of 
predicate-level schemas and the reasoning program 
for their implementation, see table 11.3 in Braine and 
O’Brien (1998). I present the following schema for a uni- 
versally quantified disjunction to illustrate the nature of 
the predicate-level schemas and to make the point that 
even though the notation is more complex than at the 
sentential level, the inferences are not necessarily more 
difficult. 


S, [All X] OR S,[PRo-All X]; [a] € [X]; NEG S,[a] .”. S;[a]. 


For example, “All of the boys from the village are at the 
river fishing or in the forest hunting. Uira and Wera are 
boys from the village and they are not at the river fish- 
ing, so they are in the forest hunting.” Unlike standard 
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logic, which suggests that reasoning at the predicate 
level is intrinsically more difficult than at the sentential 
level, this example illustrates that intuitively, this is not 
the case in natural logic. 


5. Pragmatic Principles 


People do not reason only by applying natural-logic sche- 
mas. An intelligent person uses any information that is 
pertinent, and inferences thus can be made on the basis 
of many kinds of processes. A person should interpret 
words in plausible ways and not limit their interpreta- 
tions to what a professional logician has in mind. If, for 
example, can invite additional sorts of interpretations 
(Geis & Zwicky, 1971)—for example, “If p then q” can 
invite interpretation as a biconditional, depending on 
context; if I am told that if I paint the shed I will receive 
$50, I expect that if I do not paint the shed, I will not 
receive the money. Natural logic does not constrain or 
as either exclusive or inclusive, but certain kinds of situ- 
ations surely invite one interpretation over the other, 
which can expand the available inferences. 

Story grammars or scripts provide additional infer- 
ences that a rational person would make. My knowledge 
about eating breakfast in a typical American diner tells 
me that I will not drink both tea and coffee, and this 
knowledge provided by a restaurant script can feed a 
natural-logic inference. Although a cognitive scientist 
is interested in the sources from which inferences are 
made, people are unlikely to attend to the varied sources 
that can lead to an inference or to distinguish purely 
logical inference from other sources. These things are 
not evidence of irrationality; in ordinary reasoning, 
inferences from various sources intertwine with infer- 
ences from natural logic to provide reasoning that meets 
practical concerns as lines of reasoning are constructed. 


6. Empirical Support for the Braine—O’Brien 
Natural Logic 


The most basic prediction of natural logic is that the 
inferences available on the DRR alone will be made rou- 
tinely unless something complicates matters, and the 
evidence, generally from problems presenting neutral 
content, supports this prediction. Further, inferences 
that are valid in standard logic but not available on the 
schemas of natural logic are made far less often, as are 
inferences requiring strategies that go beyond what is 
available on the DRR alone. Finally, problem length per 
se does not predict problem solution, which depends 
for the most part on whether the required inferences 
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are available in the DRR (Braine et al., 1984; Braine & 
O’Brien, 1998; see review in O’Brien, 2004). 

Braine et al. (1984), with sentential problems, and 
Yang, Braine, and O’Brien (1998), with predicate-level 
problems, presented DRR problems and asked people 
to rate on a Likert-type scale the relative difficulty of 
each problem. Regression analyses provided weights for 
each schema. Problem difficulty was assumed to equal 
the sum of the difficulty weights for all of the schemas 
required to solve a problem. These weights were used 
to compute predicted difficulties for different problems 
that were presented to different people. The percentage 
of variance accounted for in correlations between the 
predicted and observed difficulties was 66% for senten- 
tial problems (55% with problem length partialed out) 
and 69% for predicate-level problems (56% with prob- 
lem length partialed out). This indicates that people 
were not only solving these problems but also solving 
them using the natural-logic schemas. 

When the schemas were embedded within story sce- 
narios and readers were asked whether the natural-logic 
schemas were inferences or paraphrases, people showed 
that they did not know that they were making infer- 
ences at all, although inferences based on story scripts, 
grammars, and so forth were judged as being inferences 
rather than paraphrases (Lea, 1995; Lea, O’Brien, Fisch, 
Noveck, & Braine, 1990; O’Brien, Roazzi, Dias, & Sos- 
kova, 2007). Thus, the natural-logic inferences were 
made so easily that people did not think of them as 
requiring reasoning. 

Braine et al. (1995) and O’Brien, Braine, and Yang 
(1994) presented problems with multiple premises and 
asked participants to write down all of their inferences 
in the order they made them. Consider the following 
problem with five premises from O’Brien et al. (1994) 
that referred to some letters written on a blackboard: (a) 
“N or P,” (b) “Not N,” (c) “If P then H,” (d) “If H then 
Z,” and (e) “Not both Z and S.” Premises (a) and (b) trig- 
ger a logic schema to infer P, which then combines with 
(c) to infer H, which then combines with (d) to infer Z, 
which then combines with (e) to infer not-S. So, as each 
premise was read in turn, a schema was triggered that 
provided an inference. A matched problem presented 
the same premises but with the order reversed, so that 
no schema was triggered until all premises had been 
read and the information in the last two premises could 
trigger a schema. For both problems, the same infer- 
ences were made in the same order, demonstrating that 
the order of inferences was determined by availability 
of schemas. In summary, the predictions about which 
problems will be solved, about the relative difficulties 
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with which problems are perceived, and about the orders 
in which inferences are made are supported by the avail- 
able data. Further, when the predictions differ from 
what would be predicted by a standard textbook logic, 
the data support the predictions of natural logic. Sup- 
porting evidence that the core inferences are available in 
children is presented in Braine and Rumain (1983) and 
Braine and O’Brien (1998). 


7. Why Theories of Reasoning Need a Natural Logic 


Proponents of several other theories have argued against 
any role for a natural logic. O’Brien and Manfrinati 
(2010) and O’Brien and Li (2013) have provided detailed 
discussions to counter their arguments against a natural 
logic. I concentrate here on whether those theories are 
adequate without including a natural logic. 

Cheng and Holyoak (1985) introduced some content- 
specific inference rules for permissions and obligations 
that they proposed are inductively acquired, and Cos- 
mides (1989) introduced some social-contract rules that 
she proposed have been acquired through our bioevo- 
lutionary history. Both Cheng and Holyoak (1985) and 
Cosmides (1989) argue that people do not use a natural 
logic but only content-dependent processes. Both theo- 
ries rely on variants of Wason’s selection task, compar- 
ing performance with universally quantified indicative 
conditionals to performance with sentences that should 
trigger either social-contract rules or pragmatic schemas. 
The problems required people to identify situations that 
could falsify the indicative conditionals or find rule vio- 
lators of the pragmatic or social-contract rules, that is, 
instances of “p and not-q.” Both theories reported more 
correct answers with their content, which they argued 
indicates that people do not reason with a natural logic 
but with their content-specific processes. 

I address here whether these two theories provide an 
adequate account of human reasoning without includ- 
ing a natural logic, given that they specifically argue 
against any natural logic. (O’Brien & Manfrinati [2010] 
and O’Brien, Roazzi, Athias, & Brandao [2007] provide 
more extensive discussions of difficulties in interpret- 
ing the data for these content-specific theories.) The 
contention of both theories that reasoning relies not on 
a content-general natural logic but on some content- 
specific processes is based on very little evidence. They 
have investigated only a handful of conditionals that 
express permissions, obligations, or social contracts, and 
to support the notion that content-specific processes are 
sufficient to replace natural logic, they would need to 
provide a much larger set of types of content. I know 
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of no metric with which to estimate the number of 
content areas that one would need to provide a general 
reasoning theory, but it must be immense, and I find it 
hard to imagine that either inductive or bioevolution- 
ary methods would have provided such a number of 
content-dependent modules. Further, they would need 
to present evidence beyond a sole reliance on Wason’s 
selection task. 

O’Brien, Roazzi, Dias, Cantor, and Brooks (2004) 
and O’Brien (1995) discussed why data from Wason’s 
selection task are difficult to interpret, and there is 
little discussion by critics of natural logic of exactly 
how one would solve the task using logic. The selection 
task actually is a meta-logical task when presented with 
indicative conditionals—it requires judging the condi- 
tions under which the truth of a conditional could be 
tested, so a logic inference alone would not be sufficient, 
whereas the task presented with pragmatic or social- 
contract conditionals and requiring only identification 
of rule violators is not a meta-logical task. No attention 
has been given to this difference by proponents of these 
theories. If these content-specific theories are to expand 
their focus, they will need to present problems other 
than the selection task. 

Finally, these theories need to explain why if-sentences 
are used to convey these regulations. Let me illustrate the 
problem by pointing out an implicit difference in devel- 
opmental expectations between the pragmatic-schemas 
approach and the natural-logic approach. If one assumes 
that the pragmatic rules are inductively acquired, as 
pragmatic-schemas theory does, then a content-general 
schema of the sort we describe would enter late in the 
process only, after many pragmatic schemas had been 
acquired, yet Braine and O’Brien (1998) present evidence 
from spontaneous speech that young children grasp the 
schemas for conditionals quite early. 

Oaksford and Chater (e.g., 2010) proposed that the 
probability that someone will accept an inference from 
a conditional premise depends on their conditional 
belief in the conclusion given the premises and other 
background information. They make several arguments 
against natural logic, but their arguments, especially 
their arguments about defeasibility, stem from treat- 
ing natural logic as equivalent to standard logic. More 
important is the question of whether Oaksford and 
Chater are able to account for the data without includ- 
ing a natural logic. Their approach depends on prior 
beliefs about the premises on which inferences rely, 
so the approach is mute when people receive premises 
about which they have no prior beliefs. The approach 
thus cannot address the large body of evidence in favor 
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on the natural-logic approach that refers to materials 
about which participants have no prior beliefs, like the 
size and colors of beads in an urn or letters written on 
an imaginary blackboard. Further, Oaksford and Chater 
have addressed only Wason’s selection task and the four 
conditional syllogisms, and they need to explain the 
larger set of problems that we have investigated, which 
include conjunctions, negations, disjunctions, and con- 
ditional conclusions. 

Evans and Over (2004) and Over and Evans (2003) 
proposed that people evaluate conditionals by imagin- 
ing two situations, one in which both p and q are true 
and one in which p is true and q is false, and then assess- 
ing the probabilities of the two cases, and the probabil- 
ity of “If p then q” is assigned from the comparison of 
probabilities for the two models. So, “If p then q” is func- 
tionally equivalent to P(q |p) computed from the prob- 
abilities assigned to the two models. 

This proposal bears a clear resemblance to the natural- 
logic schema for conditional proof in that it begins with 
the supposition of p in constructing the two models in 
which p is true and then reasons toward gq or not-q, but 
Evans and Over (2004) argued that because the natural- 
logic approach does not include either a possible-worlds 
semantics or a mental-models semantics, it cannot account 
for these probability judgments. Why one would need a 
possible-worlds or a mental-models semantics to judge 
these probabilities is not clear. 

To illustrate that one does not need a separate logical 
semantics to make such judgments, imagine two boxes, 
a red box with 80 beads and a blue box with 20 beads. 
You are told that I have a bead in my hand and are asked 
to judge the probability that if the bead came from one of 
these boxes, it came from the red box. Using the natural- 
logic procedures for the conditional-proof schema, you 
begin by supposing that the bead came from one of 
these boxes, and you reason toward the proposition that 
it came from the red box. You discover that you cannot 
derive whether or not it came from the red box, so you 
know that the conditional cannot be shown to be true 
or false, but given that there are four times more balls 
in the red box and you are asked to judge the probabil- 
ity of the conditional, you interpret it as asking what is 
the probability that the ball came from the red box and 
respond that the probability is .80. No independent logi- 
cal semantics of any other kind was required for some- 
one to give such a response. 

The theory of Evans and Over is not adequate to 
account for reasoning in general without a natural logic 
because their theory has not been developed beyond a 
single kind of judgment about conditionals and thus 
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cannot provide an account of the larger set of findings 
concerning other connectives. It has the advantage over 
the content-dependent theories because it is not tied to 
a single content domain, and it obviously bears a close 
resemblance to the Braine—O’Brien natural-logic theory. 
Integrating their proposal with a natural logic thus seems 
eminently possible, and such an integration clearly 
needs to happen if their theory is to address a more gen- 
eral class of reasoning types, although the natural-logic 
theory seems already capable of making the judgments 
their theory describes. 

I turn finally to mental-models theory (e.g., Johnson- 
Laird, 1983; Johnson-Laird & Byrne, 1991, 2002)—the 
only other theory to address the same broad class of 
reasoning problems that natural logic addresses—whose 
proponents consistently deny that there is any role in 
human reasoning for any natural logic. The models 
theory proposes that humans construct models that 
are adaptations of truth tables but represent not truth 
assignments but possible states of affairs. Because work- 
ing memory is limited, people usually construct only 
some of the possible states of affairs. Although a com- 
plete representation of “If p then q” includes the three 
models corresponding to the material conditional, it 
usually is represented with 


[pl] 4 


where the ellipsis acts as a reminder that additional 
models could be constructed, and the brackets around 
p—called “exhaustivity markers”—constrain any such 
additional models so as not to include a token for p with- 
out a token for q. 

If the minor premise for a modus ponens argument is 
added to the model for “If p then q,” one gets 


pla Pp pa 


This set of models separates models for the two prem- 
ises and then their combination with semicolons, and 
the conclusion, q, is “read off” from the final model and 
does not include p presumably because pragmatics pre- 
cludes inclusion of categorical premises in a conclusion. 
In this way, they assert, one can derive a modus ponens 
conclusion with only the manipulation of models and 
no logic. 

Precluding any logic from their theory, including the 
variables in a predicate-argument structure, is a signifi- 
cant loss. Johnson-Laird and Byrne (1993) wrote that 
although models do not contain variables, variables do 
occur “in the initial semantic representations” (p. 376), 
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that is, the representations from which they say their 
models are constructed. 

Models for quantified propositions are almost identi- 
cal to those for the sentential connective, and the com- 
plete model for “All P are Q” is 


[ 
[ 
[ 
[ 


Let’s consider how models come from the premodels 
representations. Quantifiers provide “the raw material 
for a recursive loop that is used in building or manipu- 
lating a model. Thus, the universal quantifier ‘all’ elicits 
a recursion that deals with a set exhaustively, whereas 
the existential quantifier ‘some’ elicits a recursion that 
does not” (Johnson-Laird & Byrne, 1991, p. 178), which 
is illustrated with the example “All x’s are equal to the 
sum of some y and some z,” which is parsed to yield 


(All x) (Some y) (Some Z) (x = y+Z). 


A model, then, is constructed with an arbitrary value 
for the first variable term (x), another arbitrary value for 
the second (y), and a constrained value (the degrees of 
freedom being exhausted) for the third variable (z), loop- 
ing over the equation several times, recording the out- 
put each time, resulting in a model like 


[8 6] (16 42) (772244), 


where the two numbers in the first set are equal to some 
number in the second set plus some number in the third 
set. In this, Johnson-Laird and Byrne have succeeded in 
their goal of constructing a representation that has no 
logical variables or quantifiers, but at what cost? 

They have told us repeatedly that conclusions are 
“read off” from final models, so what conclusion could 
be read off here? Consider the simplest logic inference— 
reiteration: p ..p. But here, the meaning of the initial 
representation prior to construction of the model is gone, 
and one has no way of retrieving “All x’s are equal to the 
sum of some y and some z” from the final model. This 
is a steep cost to fulfill the goal of creating a reasoning 
theory that requires no logic inferences and no variables 
and only the manipulation of models. It’s difficult to see 
that the procedure is analogous to the standard inter- 
pretation of quantifiers in the predicate calculus. The 
quantifiers and variables of the initial proposition have 
disappeared, and without them, one cannot retrieve 
the assertion from the model. Clearly, mathematical 
reasoning cannot be accomplished purely through the 
manipulation of models. How could one use this sort 
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of procedure to prove “All natural numbers that end in 
zero are divisible by five” when one is limited to inclu- 
sion only of individual exemplars, square brackets to 
mark exhaustivity, and some ellipses to remind one that 
additional models might need to be constructed? Yet my 
students in undergraduate classes over many years have 
succeeded in understanding the necessary truth of this 
statement. 


8. Conclusion 


I have argued that an adequate account of human rea- 
soning must include some predicate-argument struc- 
ture and some inference processes for going beyond the 
information given and that other ways of thinking need 
to be integrated with the natural logic. Searching for 
counterexamples as one does with mental models would 
be a valuable skill to add to what is provided by natural 
logic, as would be the specialized searches for violators 
of pragmatic and social-contract rules. But to focus only 
on such specialized skills at the expense of the more gen- 
eral architecture provided by a natural logic would be to 
throw out the baby with the bathwater. 
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Summary 


Psychologists have studied syllogistic inferences for 
more than a century, because they can serve as a micro- 
cosm of human rationality. “Syllogisms” is a term that 
refers to a set of 64 reasoning arguments, each of which 
comprises two premises, such as, “All of the designers are 
women. Some of the women are not employees. What, 
if anything, follows?” People make systematic mistakes 
on such problems, and they appear to reason using dif- 
ferent strategies. A meta-analysis showed that many 
existing theories fail to explain such patterns. To address 
the limitations of previous accounts, two recent theories 
synthesized both heuristic and deliberative processing. 
This chapter reviews both accounts and addresses their 
strengths. It concludes by arguing that if syllogistic rea- 
soning serves as a sensible microcosm of rationality, the 
synthesized theories may provide directions on how to 
resolve broader conflicts that vex psychologists of rea- 
soning and human thinking. 


1. A Microcosm of Rationality 


In 1908, the German scholar Gustav Stdrring published 
a 130-page manuscript detailing the results of the first 
known experiments on human reasoning. His main 
purpose in conducting them was to develop solutions 
to long-standing debates between logicians and philos- 
ophers, such as what people imagine when they reason. 
The studies worked like this: volunteers entered a dark 
room alone, sat down, and received a battery of deduc- 
tive reasoning problems called “syllogisms,” one after 
another. Stérring recorded his observations of their ver- 
bal responses, reaction times, eye movements, gestures, 
and even their breathing patterns (Storring, 1908). 

The research would likely be rejected were it to be 
submitted to any contemporary psychology journal. For 
one thing, St6rring investigated only four participants. 
For another, he used an arbitrary experimental design, 


and he failed to present any quantitative analysis of 
their behaviors except for a single table that listed aver- 
aged reaction times. But what Storring learned from his 
research was remarkable (see Clark, 1922; Knauff, 2013; 
Politzer, 2004). He noticed, for instance, that his volun- 
teers were biased by the structure of the different rea- 
soning problems: for some syllogisms, volunteers chose 
conclusions immediately, as though they could observe 
the answer directly. For other problems, they reported 
the sensation of Nachdenken, that is, “a feeling of delib- 
eration.” His volunteers appeared to adopt certain strate- 
gies as they carried out the task, and they seemed aware 
of their strategies well enough that they could articu- 
late them. They reported that they used their imagina- 
tion and mental imagery on many problems, and when 
probed further, they were able to depict that experience 
by sketching out corresponding diagrams. Perhaps the 
most important discovery was that Storring’s volunteers 
had a limited ability to select logically valid conclusions, 
that is, conclusions that were true in all the situations in 
which the premises were true (cf. Jeffrey, 1981, p. 1)—for 
some problems, they produced correct answers, and for 
others, they made mistakes. 

Scientists often make use of microcosms as a way of 
understanding broader phenomena. For instance, the 
geneticist Gregor Mendel examined pea plants to under- 
stand genetic inheritance; the entomologist Agostino 
Bassi studied silkworms to understand bacterial diseases. 
In his experiments on syllogisms, Stérring had analyzed 
a feasible microcosm of human rationality. In the years 
that followed, syllogisms played an outsized role in edu- 
cating contemporary researchers on the processes of 
thinking and reasoning. Many experiments investigated 
syllogistic reasoning in isolation, and many more used 
syllogistic reasoning as a stand-in for reasoning behav- 
ior more generally. For instance, Goel and colleagues 
ran a neuroimaging experiment in which they gave 
participants syllogistic reasoning problems with and 
without meaningful contents to discover that certain 
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brain regions—such as temporal and frontal regions— 
systematically respond to semantic information (Goel, 
Buchel, Frith, & Dolan, 2000). 

Perhaps syllogisms serve as an attractive microcosm 
of thinking behavior because of their simplicity and that 
there is only a finite number of them. Classical syllo- 
gisms (i.e., those investigated by Aristotle and Scholastic 
logicians) are reasoning arguments comprising multiple 
premises, such as 


(1) All of the women are designers. 

Some of the employees are not women. 

What, if anything, follows? 
These syllogisms contain a quantified noun phrase, such 
as “all of the women,” and these quantifiers can be in one 


of four separate moods, that is, expressions comprising 
quantifiers and negations, as shown below: 


All of the a are b. (Aab) 
Some of the a are b. (lab) 

Some of the a are not b. (Oab) 
None of the a is b. (Eab) 


The parentheses indicate the abbreviation conventions 
adopted by Scholastic logicians (i.e., the 12th-century uni- 
versity scholars who gained access to Aristotle’s works). 
Contemporary psychologists adopted those conven- 
tions, and we retain them here. Since syllogisms consist 
of two premises, the terms in the premises (e.g., “women,” 
“designers,” “employees”) can occur in four different 
arrangements. These different arrangements are known as 


figures: 


Figure 1 Figure2 Figure3 Figure 4 
a-b b-a a-b b-a 
b-c c-b c-b b-c 


Other psychologists use different numbering systems for 
figures—and they sometimes include the conclusion as 
part of their numbering systems. Here we state the fig- 
ures in terms of the premises only. 

In sum, syllogisms concern 64 separate reasoning 
problems (4 moods of the first premise x 4 moods of the 
second premise x 4 separate figures). Many experiments 
on syllogisms focus on only these 64 problems, that is, 
they provide participants with the pairs of premises and 
then ask them to infer what follows from them. Typi- 
cally, reasoners do not consider all the possible valid and 
invalid responses when they generate conclusions; they 
tend to describe just one or two. But across the problems 
overall, their conclusions can be classified into nine dif- 
ferent structural patterns, as follows: “All of the As are 
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Cs” (abbreviated as Aac); “All of the Cs are As” (abbrevi- 
ated as Aca); the responses that correspond to lac, Ica, 
Oac, Oca, Eac, and Eca; and the response that no valid 
conclusion follows. 

As Stdrring recognized, some syllogisms are quite 
easy, and others can be difficult. Consider this problem: 


(2) Some of the designers are animators. 
All of the animators are undergraduates. 


What, if anything, follows? 


Before reading further, how might you respond to the 
problem? If you inferred that some of the designers are 
undergraduates, you’d be correct. But now consider this 
problem: 


(3) None of the designers are animators. 
All of the animators are undergraduates. 


What, if anything, follows? 


This problem is significantly harder: the correct response 
is that some of the undergraduates are not designers. 
Why is (2) easy but (3) difficult? To answer the question, 
psychologists have run many studies on which infer- 
ences people conclude from the 64 syllogisms. Khemlani 
and Johnson-Laird (2012) compiled six of them together 
in a meta-analysis, which shows that the most common 
response to (2) is “Some of the designers are undergradu- 
ates” (see figure 3.3.1, left panel, row Iab Abc), and the 
most common response to (3) is “None of the design- 
ers are undergraduates” (see figure 3.3.1, left panel, row 
Eab Abc)—the response is an error, since (3) does not 
rule out the possibility that some of the designers are 
undergraduates. 

In a typical study, responses to a syllogism vary 
from one individual to the next. Consider how people 
respond to (1), that is, figure 3.3.1, right panel, row Aba 
Ocb: some draw an (erroneous) conclusion of the form 
Oca about half of the time; others make a different error 
and conclude Ica. Only about a fifth of university stu- 
dents (who typically serve as participants in studies on 
syllogisms) respond correctly that there is no valid con- 
clusion. The psychologist’s task is to explain the robust 
patterns of inference across the set of 64 problems. The 
difficulty for theorists is that reasoners approach the 
problems with different abilities and appear to develop 
different strategies (see table 3.3.1). The variability in 
reasoners’ responses was enough to convince some theo- 
rists that the only way to understand how people reason 
syllogistically is to examine their individual differences 
(Stenning & Cox, 2006)—some reasoners generate correct 
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Figure 3.3.1 
The percentages of responses to 64 syllogisms in the meta-analysis in Khemlani and Johnson-Laird (2012). Each of the 64 pairs of 
premises occurs in a row, and each of the possible responses occurs in a column. Abbreviations for premises are as follows: Adc = All of 
the A are C, lac=Some of the A are C, Eac= None of the A is a C, Oac=Some of the A are not C, and NVC = No valid conclusion. The 
left panel denotes the 27 syllogisms with a valid definite conclusion, and the right panel denotes the 37 syllogisms without a valid 
definite conclusion. The grayscale in each cell indicates the proportion of corresponding conclusions (black = 100% and white = 16% or 
below). Hence, for the top-most valid syllogism, Aab Abc, nearly 100% of participants in the meta-analysis responded that Adc follows. 
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Table 3.3.1 
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Summary of robust sources of differences in syllogistic reasoning performance 


Phenomenon Description 


Differences between problems 


Some syllogisms are extremely difficult (1% correct solutions), and others are easy 


(90% correct conclusions). Differences in difficulty are reliable across studies. 


Differences among individuals 


Accuracy varies greatly: from one reasoner to another, between certain groups of univer- 


sity students, and as a function of intelligence and training. 


The figural effect 


The figure of a syllogism (i.e., the order in which terms are organized) affects the fre- 


quency and type of conclusions reasoners tend to draw. 


The effect of contents 


Individuals are more likely to accept believable conclusions and less likely to accept 


unbelievable ones, especially for invalid syllogisms. This is also referred to as the “belief 


bias” effect. 


Developmental differences 


Children are unable to draw sensible inferences from syllogisms until they can 


understand and produce quantifiers. The ability develops through adolescence up to 


adulthood. 


Strategic differences 


Reasoners acquire various strategies to solve syllogisms (e.g., which premise is interpreted 


first, how premises are interpreted, and how counterexamples are constructed). 


answers for 85% of syllogisms, and others generate cor- 
rect answers for only 15% of them (Johnson-Laird, 1983, 
pp. 118-119). 

The study of syllogisms arguably resulted in a major 
challenge to human rationality. Early work by Wood- 
worth and Sells (1935) suggested that, instead of reason- 
ing their way through syllogisms, people were biased by 
the “atmosphere” created by the premises, which yielded 
a predisposition to accept a certain sort of conclusion. 
The atmosphere effect suggested wholesale irrational- 
ity: humans could diverge from normative reasoning 
behavior predictably and systematically (see chapter 2.3 
by Johnson-Laird, this handbook). The result sparked 
a fascination with the extent to which human reason- 
ing could be characterized as rational (see chapter 1.2 
by Evans and chapter 3.1 by Steinberger, both in this 
handbook), and theorists began to devise accounts of 
the phenomena underlying syllogistic reasoning. The 
use of computational and formal tools helped some 
researchers implement psychological theories of the syl- 
logism and test them against human data. As a result, 
after decades of research, nearly a dozen theories of the 
phenomenon had been proposed, and there existed a 
dire need to sort out the different theoretical proposals. 
Khemlani and Johnson-Laird (2012) surveyed existing 
psychological accounts of syllogistic reasoning to dis- 
cover broad trends between them. The survey suggested 
that theories tended to fall into one of three groups: 
one group of theories explained syllogistic reasoning by 
appealing to sets of heuristics in how quantified state- 
ments were processed (e.g., Begg & Denny, 1969; Chater 
& Oaksford, 1999; Revlis, 1975; Wetherick & Gilhooly, 
1995). For example, the so-called matching strategy 


(Wetherick & Gilhooly, 1995) posited that for syllogisms 
such as 


(4) Some of the designers are women. 
Some of the women are employees. 


What, if anything, follows? 


people should conclude—erroneously—that “some of the 
designers are employees.” The reason is because the con- 
clusion matches the mood of the most “conservative” 
premise, that is, the premise that presupposes the exis- 
tence of the fewest entities. And indeed, reasoners draw 
the predicted conclusion 61% of the time (see figure 3.3.1, 
right panel, row lab Ibc). But about a third of the time, 
they also accurately infer that “No valid conclusion” fol- 
lows, and accounts based on heuristics have difficulty 
explaining the deliberative processes by which reasoners 
correct their mistakes (see also Ragni, Dames, Brand, & 
Riesterer, 2019). In order to account for deliberative rea- 
soning, another group of psychological theories proposed 
that reasoners mentally simulate the situation described 
in the premises when they reason about syllogisms (Buc- 
ciarelli & Johnson-Laird, 1999; Guyote & Sternberg, 1981; 
Johnson-Laird & Steedman, 1978; Polk & Newell, 1995). 
The theories posited that mental simulations help explain 
both errors and correct responses: reasoners construct, 
and can make inferences from, initial simulations, but 
difficult syllogisms demand reasoners to consider alter- 
native simulations (Johnson-Laird, 1983). A third group 
of theories assumed that syllogistic reasoning depends 
on mental proofs and rules of inference akin to those in 
formal logic (see, e.g., Braine & Rumain, 1983; Geurts, 
2003; Politzer, 2007; Rips, 1994)—but such theories have 
systematic difficulty explaining how reasoners draw the 
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conclusion that nothing follows from a set of premises, 
and so we presently address only the first two groups of 
theories. 

Theories based on heuristics and theories based on 
deliberation both failed to explain many systematic 
patterns of syllogistic reasoning (Khemlani & Johnson- 
Laird, 2012). In retrospect, the debate between the two 
types of processing presents a false dichotomy. A robust 
theory of syllogistic inference needs to explain both 
heuristic and deliberative processing (see Evans & Stano- 
vich, 2013; Johnson-Laird & Steedman, 1978). The most 
recent psychological accounts of syllogistic reasoning 
have sought to unify the two kinds of reasoning pro- 
cesses. This chapter reviews these recent theories and 
their computational implementations, and it summa- 
rizes their strengths and weaknesses. It also provides a 
broader perspective on how the theories address ongo- 
ing debates in the psychology of reasoning. 


2. Unified Accounts of Syllogistic Reasoning 


An ancient idea is that human thinking relies on two 
different systems: one fast, one slow. Peter Wason, with 
his students Philip Johnson-Laird and Jonathan Evans, 
proposed that reasoning processes should be construed 
in terms of two distinct, interreliant processes (Johnson- 
Laird & Wason, 1970; Wason & Evans, 1974). As Evans 
(2008, p. 263) notes, the dichotomy between heuris- 
tics and deliberation is closely related to dual processes 
because heuristics are thought to be a fast, shallow form 
of processing and deliberation is thought to be a slower, 
deeper form of processing. In practice, heuristics and 
deliberative thinking often occur sequentially, that is, a 
heuristic response is proposed and a deliberative process 
validates or falsifies it. More general accounts of dual pro- 
cessing are not committed to sequential processing—they 
permit that fast processes and slow processes can operate 
in parallel and interact with one another. The introduc- 
tion noted that previous theories of syllogistic reasoning 
tended to account for one type of process over the other. 
It may be that previous theories were easier to formulate 
because it is difficult to anticipate the interactive effects 
of two interdependent processes. Yet, if it is indeed the 
case that human thinking depends on two inextricable 
processes, those theories were doomed to fail. 

Two recent theories of syllogistic reasoning are unique 
in that they seek to model interactive processing. Both 
theories are built around the integrative idea that fast, 
heuristic processing is the result of a biased sampling 
procedure that can be formalized using probabilistic con- 
straints and that slower, deliberative processing suggests 


229 


that reasoning depends on representations referred to as 
“mental models.” The theory that people construct mental 
models when they reason originates from Johnson-Laird 
(1983; see also chapter 2.3 by Johnson-Laird, this hand- 
book), who computationally developed earlier proposals 
that people build “small-scale models” of reality to antici- 
pate events (Craik, 1943). Johnson-Laird’s “model theory” 
posits that each mental model represents a distinct pos- 
sibility or situation in the world. In other words, when 
reasoners draw inferences from syllogisms, they mentally 
simulate the situation referred to by the premises. The 
model theory predicts that problems that require reason- 
ers to consider multiple mental models should be more 
difficult relative to those that require fewer models. As a 
result, models help explain reasoning difficulty in many 
domains (see, e.g., Johnson-Laird & Khemlani, 2013; 
Khemlani & Johnson-Laird, 2017). But, as Khemlani and 
Johnson-Laird (2012) show, previous implementations of 
the model theory tend to make overly liberal predictions 
of the kinds of syllogistic inferences people are likely to 
draw. Hence, the two latest theories of syllogistic reason- 
ing add additional constraints that explain why reason- 
ers are reticent to draw overly liberal conclusions from 
model-based representations. We describe each theory 
in turn. 


2.1. The Probability Sampling Model 

Masasi Hattori developed a recent account of syllogis- 
tic reasoning called the “probability sampling model” 
(PSM; Hattori, 2016). The account holds that reasoners 
interpret a set of syllogistic premises by constructing 
a prototypical representation of them (referred to as a 
“probability prototype model”). The probability pro- 
totype model uses circles to represent set-membership 
relations, and so it is closely related to, for example, 
Euler circles and Venn diagrams (see the meta-analysis 
in Khemlani & Johnson-Laird, 2012, which reviews 
other theories based on such diagrammatic systems, and 
also chapter 13.1 by Jamnik, this handbook). The vari- 
ous intersections of the circles in the prototype model 
denote different kinds of individuals. For example, 
a diagram of “All of the designers are women” would 
include a circle representing the set of designers embed- 
ded inside a circle representing the set of women. The 
intersections of the circles represent the different kinds 
of individuals consistent with the premise, for example, 
that one area would represent the designers who are also 
women, and another would represent women who are 
not designers. Hattori posits that reasoners annotate the 
different areas of the prototype model with information 
about the probability of their occurrence. 
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Once the probabilities are established, reasoners draw 
a finite set of random samples to construct a mental 
model, that is, a small set of tokens that denote the enti- 
ties referred to in the premises. Hattori’s adaptation of 
mental model theory is more restricted than previous 
accounts. For example, provided that the two kinds of 
areas established by a prototype model have equal prob- 
ability, asample mental model of “All of the designers are 
women” can be represented in the following diagram: 


designer woman 
- designer woman 
designer woman 
- designer woman 


Each row of the diagram depicts the results of a random 
draw from the prototype model, and so the first row 
depicts a designer who is also a woman. The “~” denotes 
the symbol for negation, and so the second row depicts a 
woman who is not a designer. The additional rows depict 
additional random draws from the prototype model. In 
the PSM, the establishment of a sample mental model 
is the central representation on which a unitary reason- 
ing process operates. The algorithm works by applying 
a series of tests, one after another, to the sample mental 
model in order to generate a conclusion. Figure 3.3.2 
provides a schematic of how the full theory works. 

To test the theory’s predictions, Hattori implemented 
the theory computationally and then ran simulations 
that compared the theory’s predictions against eight 
separate data sets on syllogistic reasoning. He also com- 
pared the PSM’s predictions against two other theo- 
ries: Chater and Oaksford’s probability heuristics model 
(Chater & Oaksford, 1999) and a parameterized version 
of mental model theory (Hattori’s implementation of 
Johnson-Laird & Bara, 1984). His analyses show that the 
PSM matches the performance of both theories (Hattori, 
2016, p. 308). 

One major strength of the PSM is that it can explain 
how contents affect the kinds of inferences reasoners 
draw. For instance, consider the following problem: 


(5) Some of the Frenchmen are wine-drinkers. 
Some of the wine-drinkers are Italians. 


What, if anything, follows? 


A robust result from studies of syllogistic reasoning is 
that reasoners should be less apt to infer “Some of the 
Frenchmen are Italians,” since they likely consider it 
either implausible or rare that a person is both French 
and Italian at the same time. Hattori’s PSM can account 
for the effect as follows: the meanings of the premises 
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bias the way people construct the probability prototype 
model such that implausible areas of the representation 
are assigned low probabilities. Hattori implemented the 
process, and he described two studies that corroborate 
the predictions of his implementation (Hattori, 2016). 

However, Hattori’s theory is not without its limita- 
tions. One limitation is that the theory cannot explain 
how reasoners might draw inferences from more than 
two premises: in these cases, the two-dimensional prob- 
ability sampling model may be unable to represent all of 
the possible different individuals. Another limitation is 
that under the PSM, deliberation occurs by applying a 
set of logical tests, one after another, to a sample mental 
model. The process is akin to the way heuristic processing 
operates in previous theories (e.g., Chater & Oaksford, 
1999). No evidence at present suggests that reasoners 
apply such tests in a fixed, systematic order for each 
and every syllogism; indeed, the evidence suggests that 
reasoners tend to develop strategies over the course of a 
study on syllogisms, which would seem to conflict with 
the PSM. Moreover, running such tests in a fixed order is 
bound to produce a conclusion for any sample mental 
model—and so a clear consequence of the tests is that the 
PSM cannot account for why reasoners often spontane- 
ously respond that “No valid conclusion” follows (Ragni 
et al., 2019). Other accounts based on sampling and con- 
structing mental models suffer from similar issues (see, 
e.g., Tessler & Goodman, 2014). In general, it is too tax- 
ing for people to constantly and repeatedly apply the 
same set of tests to the representations they construct, 
and so the algorithm, although tractable, is limited and 
not cognitively plausible. In essence, the PSM does not 
explain the psychological processes that underlie how 
reasoners deliberate on syllogistic inferences. 

The primary theoretical contribution of Hattori’s 
(2016) probabilistic sampling model is that it integrates 
probabilistic machinery and sampling procedures with 
the construction of mental models. Another recent 
computational theory—mReasoner—likewise integrates 
probabilistic sampling with procedures with the con- 
struction of mental models. 


2.2 mReasoner: A Unified Computational 
Implementation of Mental Model Theory 

mReasoner (Khemlani & Johnson-Laird, 2021) is a unified 
computational implementation of mental model theory, 
which posits that reasoning depends on the construc- 
tion and manipulation of mental models, that is, iconic 
simulations of possibilities (Bucciarelli & Johnson-Laird, 
1999; Johnson-Laird, 1983; Johnson-Laird, Khemlani, & 
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Step 1: Probability Prototype Model (PPM) 
To construct a PPM that is not logically 
inconsistent with two premises. 


All P are M 
Some S are M 


Step 2: Sample Mental Model (SMM) 
To construct a SMM generating n data in 
accord with the PPM. 


Random sampling 


S&M&P 
S&M&—P 
S&-M&P 

S&-M&-P 
“S&M&P 
-“S&M&—-P 
—“S&-M&P 
S&-M&-P 


(sample generation) 


Step 3: Logical Conclusion 
To derive a logical conclusion that the SSM 
fulfills. 


35 


1. “Some S are P’”? — Any individual that is S&P? — “NO” 
2. “No S are P’?? > No individual that is S&P? — “YES”— OUTPUT 


Figure 3.3.2 
A schematic diagram of how Hattori’s (2016) probability sampling model draws syllogistic conclusions. (Used with permission 


from Hattori, 2016.) 


Goodwin, 2015a). The theory and its implementation 
are based on three fundamental assumptions: 


Mental models represent possibilities: a given asser- 
tion refers to a set of discrete possibilities that are 
observed or imagined (Johnson-Laird, 1983). 

Iconicity and discreteness: mental models are iconic, 
that is, their structures mirror the structures of what 
they represent (see Peirce, 1931-1958, vol. 4). Models 


can also include abstract symbols, for example, the 
symbol for negation (Khemlani, Orenes, & Johnson- 
Laird, 2012). They are discrete in that they do not 
consist of continuous spaces, areas, and regions. 

Dual processes: reasoning, including syllogisms, is 
based on two interacting sets of processes. Rapid, heu- 
ristic reasoning occurs as a consequence of building 
and scanning a single model. Deliberative reasoning, 
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by contrast, occurs as a result of revising the initial 
model to search for alternative models of heuristic 
conclusions (Khemlani & Johnson-Laird, 2013, 2016; 
Khemlani, Lotstein, Trafton, & Johnson-Laird, 2015). 


The computational model makes syllogistic inferences 
by stochastically constructing a mental model directly 
from the premises. Hence, unlike the PSM, mReasoner 
eschews any intermediary representations (e.g., the 
probability prototype model). Two factors dictate how 
initial models are constructed: the size of a model (i.e., 
the maximum number of entities it represents) and the 
contents of a model. One parameter in the system con- 
trols the size of a mental model. It does so by basing 
the size on a sample drawn from a Poisson distribution. 
Another parameter governs the model’s contents, which 
are drawn from the most common set of possibilities 
corresponding to a particular assertion (the canonical 
set) or else the complete set of possibilities consistent 
with the assertion. For example, in the case of “All of the 
designers are women,” reasoners tend to consider only 
one canonical possibility: female designers. But the com- 
plete set of possibilities allows for women who are not 
designers, and a parameter in the system sets the prob- 
ability of drawing from the complete set. 

To illustrate how the system makes heuristic infer- 
ences, consider this syllogism: 


(6) All of the designers are women. 
Some of the women are not employees. 
What, if anything, follows? 
Suppose that the premises for (6) are input into mRea- 


soner. The system may construct the following initial 
model: 


designer woman 

designer woman 

designer woman employee 
employee 


The diagram is similar to that of the sample mental 
model illustrated in the previous section: its rows denote 
separate individuals. To generate a heuristic conclusion, 
the system scans the model in the direction in which it 
was built. In the model above, for instance, the system 
builds tokens for “designers” first, “women” second, and 
“employees” third. Hence, the program draws an initial 
conclusion that interrelates “designers” to “employ- 
ees” (e.g., some of the designers are not employees). 
This conclusion matches the preponderance of conclu- 
sions that reasoners spontaneously generate. For other 
sorts of syllogisms, the system draws initial intuitive 
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conclusions that interrelate “employees” to “designers,” 
again depending on how the model was constructed. 

Because the heuristic conclusion depends on just a 
single model, the system generates it quickly. But, as the 
example illustrates, the conclusion may be invalid. To 
correct the error, the program can call on a deliberative 
component to search for counterexamples to conclu- 
sions (Khemlani & Johnson-Laird, 2013). It operates by 
modifying the initial model, using a finite set of search 
strategies (Bucciarelli & Johnson-Laird, 1999). When the 
deliberative system is engaged, it can find a counter- 
example to the conclusion that some of the designers 
are not employees: 


designer woman employee 

designer woman employee 

designer woman employee 
woman 


This model is one that represents the premises in (6) but 
falsifies the heuristic conclusion, and so the program 
responds that no valid conclusion holds. A separate 
parameter controls the probability that the deliberative 
system is engaged. Figure 3.3.3 provides a schematic dia- 
gram of the system’s architecture. 

mReasoner provides a close fit to the results from 
the data presented in the meta-analysis on syllogistic 
reasoning compared to alternative theories (Khemlani 
& Johnson-Laird, 2013). A major strength of the theory 
is that it is flexible enough to simulate reasoning about 
problems that include any number of premises, for 
example, one-premise “immediate” inferences (Khem- 
lani et al., 2015), two-premise syllogisms (Khemlani & 
Johnson-Laird, 2016) and set-membership inferences 
(Khemlani, Lotstein, & Johnson-Laird, 2014), and more 
complex problems composed of three premises (Ragni, 
Khemlani, & Johnson-Laird, 2014). It does so by rely- 
ing on a limited set of parameters that provide motivated 
constraints on how the theory’s predictions vary (see fig- 
ure 3.3.3). The flexibility of the system allows it to model 
individual differences and strategies in syllogistic reason- 
ing (Bacon, Handley, & Newstead, 2003; Khemlani & 
Johnson-Laird, 2016; Ragni, Riesterer, Khemlani, & 
Johnson-Laird, 2018). 

mReasoner has several limitations. One limitation is 
that at present, unlike the probability sampling model, 
mReasoner does not have machinery capable of explain- 
ing effects of content (i.e., the “belief bias” effect). In 
principle, the machinery is feasible, and a prototype 
system implements “semantic modulation” for senten- 
tial reasoning (see Khemlani, Byrne, & Johnson-Laird, 
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Figure 3.3.3 


A schematic diagram of the program implemented in mReasoner for reasoning with mental models of quantified assertions such as 


“All the designers are women.” White boxes denote procedures, gray boxes denote parameters, and the black box denotes the point 


at which the program generates a conclusion. The program integrates two separate reasoning pipelines: a fast, heuristic process 


(represented by solid arrows) that does not revise models and a slow, recursive, deliberative process (represented by dotted arrows) 


that revises initial models and draws conclusions from the set of alternative models. The diagram also highlights the stochastic 


algorithm that controls how models are built. 


2018). But the system has not been applied to simu- 
late belief bias effects. A second limitation of the the- 
ory is that, unlike theories based on Bayesian updating 
mechanisms (e.g., Chater & Oaksford, 1999; Oaksford & 
Chater, 2009), mReasoner does not explain how learn- 
ing and reasoning interact with one another. 


2.3 Integrated Models of Reasoning 

The two newest theories of syllogistic reasoning described 
here were developed to achieve parsimony relative to 
previous theories. They share many commonalities: they 
both use sampling procedures to build up representa- 
tions. They both result in the construction of mental 
models (i.e., mental simulations of possibilities). They’re 
both implemented computationally, and so their limita- 
tions are clearly defined. Because of the clarity afforded 
by their computational implementations, both have 
been validated against numerous data sets—indeed, 
both accounts manage to explain far more data than any 
single previous account of syllogistic reasoning to date. 
And they predict many empirical phenomena beyond 
the narrow scope of the 64 syllogistic reasoning prob- 
lems. As such, the theories may serve as promising foun- 
dations for future research. 


3. General Discussion 


Why have psychologists spent so much time and effort 
on understanding syllogistic reasoning? One reason is 
because Aristotle had made it central to his logic; logic 
was practically synonymous with syllogistic logic until 
Frege developed the predicate calculus, when logicians 
realized that syllogisms were but a small corner of the 
space of logical analysis. Nevertheless, syllogisms pre- 
sented a feasible microcosm for study. By the middle of 
the 20th century, researchers began to recognize that 
people’s syllogistic inferences could serve as a measure of 
their general ability to think rationally. Syllogisms were 
included in verbal reasoning tasks, entrance exams, and 
aptitude tests (e.g., Ekstrom, French, Harman, & Der- 
men, 1976; Nester & Colberg, 1984). Researchers of psy- 
chiatric disorders even began to use erroneous responses 
in syllogisms as a way to measure irrational thinking, for 
example, in patients with schizophrenia (Gottesman & 
Chapman, 1960; Williams, 1964), a practice that car- 
ries on into modern psychiatry (Mujica-Parodi, Mala- 
spina, & Sackeim, 2000). Yet, no psychological account 
of syllogisms existed when St6rring conducted his initial 
investigations into people’s logical reasoning abilities. His 
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primary goal was to resolve theoretical debates among 
philosophers and logicians; hence, it was necessary to 
develop a theory of syllogisms. 

The generation of researchers who followed Stérring 
developed theories that operated under the consensus 
that errors in reasoning were the result of fallible strat- 
egies. New theories continued to flourish late into the 
century, such that by its end, theorists had proposed 
a dozen different psychological accounts of syllogis- 
tic reasoning. As theories flourished, consensus among 
researchers dissipated. Khemlani and Johnson-Laird 
(2012) noted that the failure to arrive at a single, uni- 
fied account was a catastrophe for the science of human 
thinking: “skeptics may well conclude that cognitive sci- 
ence has failed [as it] yields no consensus about a small, 
empirically tractable domain of reasoning.” 

The dozen theories of syllogistic reasoning cannot 
all be correct: they make inconsistent predictions and 
commitments. A trivial path forward might be to stitch 
together all of the theories’ predictions by disregarding 
their inconsistencies; after all, people themselves can be 
inconsistent. Different people develop different reason- 
ing strategies, and even the same person may use dif- 
ferent strategies on different occasions (see, e.g., Ragni 
et al., 2018). Alas, an amalgamated account is unlikely to 
explain the robust patterns of responses depicted in fig- 
ure 3.3.1. Worse still, it is unlikely to concern anything 
beyond the scope of the 64 syllogistic reasoning prob- 
lems, and so it cannot provide much of a guide for how 
humans reason in general. A more productive way to 
resolve the impasse is to develop a new, unified theory of 
syllogistic reasoning, one that supersedes the old ones. 

Hattori (2016) and Khemlani and Johnson-Laird 
(2013, 2016) took such an approach in their most recent 
proposals of reasoning by syllogism. Both proposals pro- 
vide a framework for explaining (a) the interpretation 
and mental representation of syllogistic premises, (b) 
what the mind computes and how it carries out infer- 
ential tasks with such assertions, (c) the differences in 
difficulty from one inference to another and common 
errors, (d) how contents affect performance, and (e) 
individual differences in performance from one person 
to another. And their theories have been implemented 
in computer programs whose operations were used to 
simulate empirical data. 

Perhaps the most fierce debate between theorists who 
study human reasoning is the challenge of integrating 
probabilistic and deductive inference (see, e.g., Baratgin 
et al., 2015; Evans & Over, 2013; Johnson-Laird, Khem- 
lani, & Goodwin, 2015a, 2015b; Khemlani, 2018). Syllo- 
gistic reasoning stands at the nexus of the debate: people 
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have the ability to draw valid syllogistic inferences, that 
is, they have the ability to reason deductively. The result 
is evident in Stdrring’s seminal studies and in numer- 
ous studies that followed. But reasoners’ deductive abili- 
ties are limited; they often draw conclusions that are 
likely—but not guaranteed—to be true, and their own 
background knowledge biases their tendency to accept 
valid deductions. Some theorists sought to resolve the 
discrepancy by developing new probabilistic interpre- 
tations of validity (Chater & Oaksford, 1999; see also 
Evans, 2012; Over, 2009), by emphasizing those aspects 
of reasoning that are easiest to formulate with proba- 
bilistic machinery (e.g., Evans & Over, 2004), and by 
discounting the relative importance of deduction in 
daily thinking (e.g., Oaksford & Chater, 2002). But such 
treatments are controversial, even among theorists who 
argue that human rationality is fundamentally probabi- 
listic (see Johnson-Laird et al., 2015a). As a result, argu- 
ments on how to integrate probability and deduction 
often result in deadlock. Just as the two theories of syllo- 
gistic reasoning reviewed in this chapter may help adju- 
dicate between heuristic and deliberative processes, they 
may also represent ways to overcome larger deadlocks in 
debates on rationality: at their core, the theories advo- 
cate limited and biased sampling processes best char- 
acterized and formalized with probabilistic constraints. 
But the outputs of the sampling processes are discrete 
mental simulations—mental models—that help explain 
deductions, inductions, and errors in reasoning. 
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Section 4 Probabilistic Reasoning 


4.1 Subjective Probability and Its Dynamics 
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Summary 


This chapter is a philosophical survey of some lead- 
ing approaches in formal epistemology in the so-called 
Bayesian tradition. According to them, a rational agent’s 
degrees of belief—credences—at a time are representable 
with probability functions. We also canvas various further 
putative “synchronic” rationality norms on credences. 
We then consider “diachronic” norms that are thought 
to constrain how credences should respond to evidence. 
We discuss some of the main lines of recent debate and 
conclude with some prospects for future research. 


1. Outline of the Chapter 


You are certain that something exists, slightly less con- 
fident that your lottery ticket will lose, less confident 
again that it will be cloudy tomorrow, still less confident 
that your lottery ticket will win, and you have no con- 
fidence whatsoever that nothing exists. Traditional epis- 
temology traffics mainly in the binary doxastic states 
of belief and knowledge, and classical logic is thought 
by many to provide norms for them (see chapter 3.1 by 
Steinberger and chapter 5.1 by van Ditmarsch, both in 
this handbook). But we also have a whole spectrum of 
belief-like attitudes. Following Ramsey (1926/1990), we 
may call them degrees of belief and regard them as being 
answerable to a “logic of partial belief”: probability the- 
ory.' This, in turn, gives that mathematical theory one of 
its main interpretations: subjective probability. 

The theory of subjective probability provides us 
with a normative, numerical model of degrees of belief. 
It uses precise numbers between O and 1 to represent 
the degree to which someone believes something. More- 
over, it formulates normative requirements that their 
degrees of beliefs have to obey in order to be perfectly 
rational. Hence, the philosophical investigation of sub- 
jective probability is less concerned with the question 
of whether people are in fact rational—this question 


is delegated to empirical psychologists (see chapter 4.5 
by Chater & Oaksford, this handbook)—and more con- 
cerned with characterizing norms of ideal rationality 
and reasoning. In this chapter, we survey a variety of 
purported rational requirements and the philosophical 
justifications that have been offered in their support— 
both synchronically (for an agent at a given time) and 
diachronically (for an agent at different times). 


2. The Probability Calculus 


To understand subjective probability, we must first 
understand probability. Kolmogorov’s axiomatization 
(1933/1950) remains the orthodox formalization of 
probability—the so-called probability calculus.” We begin 
with a “universal set” Q regarded as the set of all possi- 
bilities of interest or of all possible outcomes of a given 
random experiment. Probabilities are numerical values 
assigned by a real-valued function P to certain subsets of 
Q—events—obeying the following axioms: 


1. For all events X, P(X) =0. 
2. P(Q) = 1. 


3. For all disjoint events X and Y, P(X UY) = P(X)+P(Y). 
(Finite Additivity) 


(Nonnegativity) 


(Normalization) 


These axioms are often reformulated with events replaced 
by sentences in a formal language. Normalization becomes 


P(T) = 1, where T is any tautology. 
And Finite Additivity becomes 


for all logically incompatible sentences X and Y, 
P(X v Y) = P(X) +P(Y)2 


Conditional probability is probability relative to, or given, 
some body of evidence or information. Following Kol- 
mogorov, we symbolize “the probability of X, given Y” 
as P(X | Y) and define it as 


P(X AY) 


P(X|Y) = aa 


, if P(Y)>0. 


240 


For more on the probability calculus, see Hajek and 
Hitchcock (2016).* 

This formalism is brought to life when we interpret 
“P.” Tt might be regarded, for example, as representing 
objective chances in the world, independent of anyone’s 
opinions; these might be understood as relative frequen- 
cies, as propensities, or as falling out of the best system- 
atization of the universe—see Hajek (2019) for more 
discussion and references. However, according to the 
interpretation of interest here, “P” represents a rational 
agent’s degrees of belief, also known as degrees of confi- 
dence, or credences. For example, your credence that a roll 
of a fair die lands an even number should be 


1 1121 
P(2U4U6) = P(2)+ P(4)+ P(6) = =+54+5>= -. 
(SE) = PA) Pa eye = S 
And your credence that it lands 6, given that it lands an 
even number, should be 


%_1 
“% 3 


Going in the other direction, we might begin by assum- 
ing that a rational agent has various degrees of belief and 
ask how to represent them formally. The standard view 
among philosophers, economists, and others in the “Bayes- 
ian” tradition is that someone’s degrees of belief should be 
representable by a probability function conforming to this 
calculus. This view is known as probabilism. This requires 
some analysis: what are degrees of belief? And it requires 
some justification: why should they obey these axioms? The 
justification will be tailored to the analysis. 


3. Accounts of Subjective Probability, and Arguments 
for Probabilism 


3.1. The Betting Account 

The dominant approaches see degrees of belief as inti- 
mately tied to action and decision making. A behavior- 
ist approach identifies degrees of confidence with betting 
behavior (e.g., de Finetti, 1937/1964). Its gist is as follows: 


Your probability of E is x iff x is the price for which you 
would either buy or sell a bet that pays a unit of money if E 
and that pays nothing otherwise. 


There are various problems with this account. It assumes 
that there is a unique such price when there may be none 
or more than one. Moreover, the account is implausible 
when E is an event under your control. For example, you 
assign low probability to your impersonating a chicken 
today, but that may change if your doing so wins a $1 
bet. (If $1 does not suffice, make it $1,000,000—raising 
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the further problem that your betting behavior is sensi- 
tive to the stakes involved, while your credences are not.) 
More generally, measuring your credences with bets may 
change what those credences are, as Ramsey observed. 
(Indeed, someone offering a high-stakes bet on a prop- 
osition may make you suspicious that they have inside 
knowledge about it!) And a given betting price may be 
jointly determined by many components—the value you 
attach to money, the thrill of gambling, a public show of 
bravado, and so on—only one of which is your credence. 


3.2 The Dutch Book Argument 

Despite its limitations, many authors regard the bet- 
ting account as “fundamentally sound,” in Ramsey’s 
(1926/1990) words. As such, it is appealed to in one 
of the most important arguments for probabilism: the 
Dutch Book argument. A Dutch book is a set of bets bought 
or sold at such prices as to guarantee a net loss. We have 
the celebrated Dutch book theorem, first stated by Ramsey, 
and proven by de Finetti (1937/1964): 


If a set of (normalized) betting prices violates the probability 
calculus, then there is a Dutch book consisting of bets at 
those prices. 


So, if your betting prices violate the probability calculus, 
you are susceptible to a Dutch book. However, the argu- 
ment continues, if you are susceptible, then you are irra- 
tional. Hence, rationality requires your betting prices to 
obey the probability calculus. Identifying your credences 
with your betting prices, the argument concludes: ratio- 
nality requires your credences to obey the probability 
calculus. 

The argument is often personified with a canny bookie 
fleecing a hapless agent who has nonprobabilistic cre- 
dences. For example, suppose you assign probability % 
to the Democrats winning the next election, probability 
¥% to the Republicans winning, but only probability 0.9 
to either the Democrats or the Republicans winning. The 
bookie could sell you for 50 cents a bet that pays $1 if the 
Democrats win, sell you for 50 cents a bet that pays $1 if 
the Republicans win, and buy from you for 90 cents a bet 
that pays $1 if either of them win. Initially, you pay him 
a total of $1 and he pays you 90 cents, with you losing 10 
cents upfront. Then, come what may, the bets that you 
hold pay exactly the same as the bet that he holds, so that 
your net loss remains the same. 

Often neglected, but equally important, is the con- 
verse Dutch book theorem (Kemeny, 1955): 


If a set of betting prices (normalized to 1) obeys the prob- 
ability calculus, then there is no Dutch book consisting of 
bets at those prices. 
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This dispels the concern that even so-called coherent 
agents, whose credences obey the probability calculus, 
might be susceptible to Dutch books. Not so: coherence 
inoculates us! 

The Dutch book argument is often criticized for bring- 
ing out a pragmatic rather than an epistemic defect in an 
incoherent set of credences. Furthermore, some critics 
complain that an incoherent agent need not fall into the 
trap of being Dutch-booked—for example, you might 
just walk away instead of making the sure-loss package 
of bets. In response, Skyrms (1980), following Ramsey, 
interprets the argument as merely dramatizing an incon- 
sistency in such an agent’s evaluations—an epistemic 
defect. For more discussion, see Skyrms (1993), Vineberg 
(2001, 2016), and Hajek (2009). 


3.3 Representation Theorems 

We have noted some problems with the betting account 
of credences, which carry over to the Dutch book argu- 
ment: if your credences come apart from your betting 
prices, a defect in the latter does not entail a defect in 
the former. Let’s move on, then, to another account of 
credence and another kind of argument for probabilism 
that is based on it. We have Ramsey to thank for both. 

He presupposes an operationalist attitude to defin- 
ing such a theoretical term: “the degree of a belief... 
has no precise meaning unless we specify more exactly 
how it is to be measured” (Ramsey, 1926/1990, p. 167). 
His guiding idea is that probability “is a measurement 
of belief... qua basis of action” (Ramsey, 1926/1990, 
p. 171) and to use an agent’s preferences among gambles 
to provide this measurement. He proposes a number of 
axioms governing such preferences. Some are “consis- 
tency” assumptions, such as transitivity, while others 
are structural, aimed at streamlining the mathematics. 
Together, they allow him to calibrate a utility scale for 
the agent, a measure of how desirable various gambles 
and their outcomes are. This, in turn, determines her 
credences as ratios of differences of utilities. He goes on 
to add further axioms that guarantee their obeying the 
probability calculus. 

Savage (1954) similarly derives probabilities and utili- 
ties from preferences that are constrained by certain axi- 
oms. However, he is more concerned with the normative 
status of the axioms than Ramsey is: they constrain ratio- 
nal preferences (see also chapter 8.1 by Grtine-Yanoff, 
this handbook). Savage proves a representation theorem: 
for a given (representation of a) set of such preferences, 
there is a class of utility functions, unique up to posi- 
tive linear transformations, and a unique probability 
function that jointly “represent” the preferences in the 
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following sense: the expected utility of an option (“act”) 
is the weighted average of the utilities of its possible 
outcomes associated with a set of “states,” the weights 
provided by the probabilities of the states. A utility func- 
tion (chosen from such a class) and probability func- 
tion are said to jointly represent an agent’s preferences 
when she prefers A to B iff the expected utility of A is 
greater than that of B. Jeffrey (1965) offers an alternative 
axiomatization of rational preferences to Savage’s, and 
Bolker proves another representation theorem (cited by 
Jeffrey), which typically does not yield a unique prob- 
ability function. 

Such representation theorems undergird an argu- 
ment for probabilism: any rational agent has preferences 
that obey the axioms; as such, she is representable as 
an expected-utility maximizer, with weights interpreted 
as her credences. These credences obey Kolmogorov’s 
axioms. However, some question the normative status 
of some of the preference axioms—they are there more 
to facilitate the mathematical representation. Others 
question the step from the rational requirement of the 
agent being representable probabilistically to the ratio- 
nal requirement of her credences being probabilities. 
For example, she may also be representable nonprobabi- 
listically (see Zynda, 2000). Indeed, she may be repre- 
sentable in entirely implausible ways—for example, as a 
voodoo spirit maximizer (see Hajek, 2008). 

Representation-theorem approaches to credences allow 
and even support an anti-realist stance regarding them: 
they are merely artifacts of a representation. (Stefans- 
son [2017] defends this view.) By contrast, a more real- 
ist understanding of credences fits well with our next 
arguments for probabilism. (See Eriksson & Hajek [2007] 
for further discussion of the metaphysics of credences, 
including their view that they should be regarded as 
primitive, irreducible to anything else; see also note 2 
for references to approaches that take comparative prob- 
abilities as primitive.) 


3.4 Accuracy Arguments 
Belief is typically regarded as being a genuine mental state 
rather than merely a representational artifact. Moreover, 
it is often said to have a constitutive aim: truth. As such, 
a given belief may be evaluated on whether or not it 
achieves this aim—whether it is true or false (see also 
chapter 3.1 by Steinberger, this handbook). However, 
intermediate credences apparently cannot be said to be 
true or false. Are they nonetheless answerable to a truth 
norm? 

A thinker may assign intermediate degrees of confi- 
dence to strike a balance between the competing aims of 
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believing truths and avoiding belief in falsehoods. Ide- 
ally, the agent assigns full credence to all true proposi- 
tions and zero credence to all false propositions. When 
she assigns intermediate credences, we can measure how 
close they are to the truth. A specific class of measures, 
called scoring rules or accuracy measures, has been devel- 
oped for this purpose. 

A popular measure is the Brier score: if the agent’s cre- 
dence in some event E is x, and E actually occurs, then 
the Brier score for this credence is (1 - x)’; if E doesn’t 
actually occur, the score is (0 ~ x). We can compute the 
Brier score for an entire credence function at a particular 
world by summing the individual scores for the agent’s 
credences at that world. The closer the score is to zero, 
the better, since a lower score indicates better accuracy 
or greater closeness to the truth. The Brier score has been 
put to practical use, for example, to score the accuracy of 
weather forecasters and political analysts. A feature that 
makes the Brier score, among other scoring rules, par- 
ticularly suited for this purpose is that it is strictly proper: 
someone who is asked to report her credences expects 
to minimize the score she will receive by reporting her 
actual credences. By contrast, if we used an improper 
scoring rule, such as the absolute distance measure |1 — x| 
or |O-x|, to score the agent’s credences’ distance from 
the truth, she would sometimes be better off lying about 
her credences to improve her (own expectation of her) 
accuracy score. 

Suppose we adopt the Brier score to measure accuracy. 
Then, if an agent has nonprobabilistic credences, there is 
an alternative probabilistic credence function that is more 
accurate in every possible world. Conversely, if she has 
coherent credences, there is no alternative probabilistic 
credence assignment that is more accurate in every pos- 
sible world. Hence, all and only coherent credence func- 
tions avoid being accuracy dominated. A version of this 
mathematical result was first demonstrated by de Finetti 
(1974, pp. 87-91) and has since been developed further 
to show that it holds for additional suitable scoring rules 
besides the Brier score. The theorem is the central prem- 
ise of the accuracy dominance argument for probabilism, 
which claims that having accuracy-dominated credences 
is a rational defect, which can only be avoided by having 
probabilistic credences (for developments of the theorem 
and the argument, see Joyce, 1998; Leitgeb & Pettigrew, 
2010a, 2010b; Pettigrew, 2016). 

The accuracy dominance argument is popular because 
it doesn’t rely on pragmatic considerations—it is thought 
to be an argument for epistemic rationality that pro- 
ceeds from purely epistemic premises. However, for the 
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argument to be sound, we need to select a specific accu- 
racy measure from the class of eligible strictly proper scor- 
ing rules (Bronfman, manuscript). This has presented 
something of a problem, since it is difficult to find com- 
pelling epistemic grounds for selecting a specific accu- 
racy measure.° 


4. Other Putative Synchronic Norms 


4.1 Regularity 

“Tf it can happen, then it has some probability of hap- 
pening.” Many authors have offered some version of 
this intuitive slogan as a constraint on credences (e.g., 
Carnap, 1950; Lewis, 1980; Shimony, 1955). More for- 
mally, regularity constraints have the form 


If X is possible, then C(X) > 0, 


where C represents the credences of some rational agent. 
There are various candidates for the sense of “possibil- 
ity” here, but logical, metaphysical, and epistemic possibility 
are especially prominent. 

There are both theoretical and pragmatic arguments 
in favor of regularity. Theoretical: Your credences should 
reflect your evidence; if your evidence does not decisively 
rule out X, so that X remains epistemically possible, your 
credence should not rule it out either. Furthermore, such 
an open mind regarding X is required for you to be able 
to learn it (Lewis, 1980). Pragmatic: If you assign prob- 
ability 0 to a possible proposition X, then you are sus- 
ceptible to a semi-Dutch Book (Shimony, 1955): you will 
consider fair a bet that you could lose and that could not 
possibly win you any money. According to the betting 
interpretation, you are prepared to pay $1 for a $1 bet 
on X. This seems irrational, since at best you will break 
even, and you could lose $1. However, see Hajek (2012) 
for rebuttals of these arguments (and others). 

Furthermore, there are both theoretical and pragmatic 
arguments against regularity. Theoretical: If the space of 
possibilities is sufficiently large, regularity is mathemati- 
cally impossible. For example, if there are uncountably 
many possible outcomes, then a (real-valued) probabil- 
ity function cannot assign positive probabilities to all 
of them. Indeed, uncountably many of its assignments 
must be O (see Hajek, 2003). Pragmatic: Assuming that 
your gaining infinite expected utility is possible (as in 
Pascal’s wager and the St. Petersburg paradox°), regular- 
ity requires you to assign positive probability to your 
gaining it. This, in turn, requires you to value this pros- 
pect infinitely, however small this probability—the same 
as you would if you had it for sure (see Hajek, 2012; for 
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further opposition to regularity, see also Easwaran, 2014; 
Levi, 1989; T. Williamson, 2007). 


4.2. The Principal Principle 

We have mentioned that the “P” that Kolmogorov axio- 
matized might be interpreted objectively as chance. Lewis 
offers a further rationality constraint on credences, which 
codifies a certain kind of alignment between them and 
chances. Here is his classic statement of it (Lewis, 1980, 
p. 266): 


The Principal Principle. Let C be any reasonable initial cre- 
dence function. Let t be any time. Let x be any real num- 
ber in the unit interval. Let X be the proposition that the 
chance, at time t, of A equals x. Let E be any proposition 
compatible with X that is admissible at time t. Then 


C(A/XE) = x. 


An initial credence function is a “prior” probability 
function, representing one’s credences before one receives 
any evidence. An admissible proposition is one “whose 
impact on credence about outcomes comes entirely by 
way of credence about the chances of those outcomes” 
(Lewis, 1980, p. 272). Typically, information about the 
past is admissible, whereas information about how the 
chance process turns out is inadmissible. An instance of 
the Principal Principle for a toss of a coin after noon is: 


C(heads | chancepoon(heads) = ¥%, and another toss of 
the coin landed heads at 11:59 am) = %. 


Lewis’s talk of an “initial” credence function may 
come as a surprise—it is not your credence function or 
that of any rational agent who has learned anything. Lewis 
(1994) presents the Principal Principle differently, speak- 
ing instead of “a rational credence function for someone 
whose evidence is limited to the past and present—that is, 
for anyone who doesn’t have access to some very remark- 
able channels of information” (p. 483). This far more 
natural (nonequivalent) formulation is how the principle 
is usually understood. It is also surprising that the vast 
literature on the Principal Principle overlooks the change 
in formulation.’ Pettigrew (2016) gives an accuracy-based 
justification for the principle, as well as a comprehensive 
overview and discussion of different formulations of it. 
For a Dutch book-style argument, see Pettigrew (2018). 


4.3 The Reflection Principle 

The Principal Principle is sometimes called an expert 
principle or a deference principle, because it requires ratio- 
nal thinkers to treat a source of probabilities—the chance 
function—like an expert and to defer to its assignments. 
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Another such principle is the Reflection Principle (van 
Fraassen, 1984), which enjoins thinkers to defer to their 
own future credences. Where ¢ is a particular time, and 
t+nisa later time, 


CoA | Cra n(A) = x) =X. 


Informally stated: Given that at a later time your cre- 
dence in some claim A will be x, your credence in A 
should already be x at the current time. For example, 
suppose you are certain that once you start your car 
tomorrow morning and hear the rattling of its old 
engine, you will be highly confident that the car needs 
servicing; then you should be just as confident now that 
the car will need servicing tomorrow. 

Van Fraassen argued for the principle by showing 
that there is a Dutch book argument for it similar to 
the one for conditionalization (more on that below; 
for discussion, see also Briggs, 2009; Christensen, 1991; 
Mahtani, 2015). There is also an argument showing 
that obeying Reflection maximizes the expected accu- 
racy of one’s credences (Easwaran, 2013). However, it 
has been pointed out that, for the principle to be plau- 
sible, the range of situations in which it applies must 
be restricted. If you expect your future self to be either 
irrational or forgetful, then you should not defer to 
your future credences (Arntzenius, 2003; Christensen, 
1991; Talbott, 1991). Van Fraassen (1989, 1999) offers a 
modified formulation of the principle to address these 
types of worries. 


4.4 The Indifference Principle 

Another putative norm of rationality is the Indifference 
Principle, which applies when we either have no evi- 
dence, or our evidence does not discriminate between 
the relevant possibilities. Here’s a version of it: 


If there are n possibilities, and an agent has no evi- 
dence that discriminates between them, then the ratio- 
nal credence for her to assign to each possibility is 1/n. 


For example, suppose you are informed that you have 
been entered into some kind of lottery alongside 99 other 
participants. You know nothing about how the winner is 
determined. The Indifference Principle then advises that 
you should assign a 1/100 credence to winning. 

The indifferent credence distribution is supported by 
minimax reasoning about the accuracy of one’s cre- 
dences: it minimizes the worst-case inaccuracy of one’s 
credences (Pettigrew, 2016). It also minimizes informa- 
tion among all distributions over the possibilities. For a 
discrete probability distribution P = (pi, po, ..., pn), the 
information of P is defined as 
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> P, log, p; 


(where 0 log» O is regarded as 0). Equivalently, the indif- 
ferent distribution maximizes entropy, defined (by Shan- 
non, 1948) as 


— DP log, p;- 


Entropy is regarded as a measure of the uncertainty of 
a distribution. A distribution that concentrates probabil- 
ity 1 on a single possibility is maximally informative and 
minimally uncertain; a uniform distribution is minimally 
informative and maximally uncertain. Entropy may also 
be generalized to continuous probability distributions, and 
the Indifference Principle may be generalized to the princi- 
ple that the rational probability assignment, subject to a 
given constraint (e.g., background knowledge), is the distri- 
bution with maximum entropy that meets the constraint. 
Jaynes (2003) is a prominent advocate of this principle. 

The Indifference Principle—and its generalization—is 
controversial, however, because it sometimes gives con- 
flicting recommendations, made vivid in Bertrand’s para- 
doxes (Bertrand, 1889). Van Fraassen (1989) illustrates this 
problem. Consider a factory that produces cubes with side 
lengths up to 1 foot. What should your credence be that 
a randomly selected cube has a side length up to % foot? 
The natural answer is of course 2. But this assumes that 
side length is the relevant quantity over which we should 
uniformly distribute our credence. We might just as well 
take the cubes’ volume to be this quantity. The factory 
produces cubes with volumes up to 1 cubic foot. Then 
the question becomes how confident we should be that a 
randomly selected cube has a volume up to 4° = ¥% cubic 
foot. Now the natural answer is Ys! And we get yet another 
answer if we consider the area of the cube’s faces instead. 
Since none of these ways of dividing up the possibilities 
into symmetrical alternatives stands out as privileged, the 
Indifference Principle cannot deliver a unique answer. It’s 
a matter of debate whether this is a fatal objection to the 
principle. See Norton (2008), White (2009), and Novack 
(2010) for some defenses of the principle. 


5. Diachronic Norms 


5.1 Conditioning 

Conditionalization or conditioning is the most widely 
endorsed rule that governs how a thinker should respond 
when she becomes certain of a new piece of evidence: 


When a thinker learns a piece of evidence E with certainty 
(and learns nothing in addition), then her new credence in 
any proposition A is equal to her old conditional credence 
C(A|E). 
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For example, suppose you are 90% confident that your 
neighbor is home, given that his lights are on. Other- 
wise, you are only 50% confident that he is home. If you 
then see that his lights are on, your confidence that he is 
home should be 90%. 

Both Dutch book and accuracy arguments have been 
offered in support of conditioning. The Dutch book is 
diachronic, because it requires that bets be placed at two 
different times—before and after learning the new evi- 
dence. Lewis (1999) shows that violations of condition- 
alization lead to Dutch book vulnerability. Skyrms (1987) 
provides the needed converse theorem: conditionalizers 
are immune to Dutch books (provided they don’t commit 
any other Dutch-bookable offenses). Critics worry that 
diachronic Dutch book arguments are less compelling 
than synchronic ones. Christensen (1991) argues that 
Dutch-bookable inconsistencies between one’s earlier and 
later self’s credences are no more irrational than similarly 
Dutch-bookable disparities between one’s own credences 
and one’s spouse’s credences. Taking the betting story lit- 
erally, Levi (1987) argues that diachronic Dutch books are 
ineffective, since the agent would simply refuse the ini- 
tial bets if she were aware of the bookie’s strategy. Skyrms 
(1993) shows that once the decision faced by the agent is 
fully specified, she can be enticed to bet—and lose—even 
if she knows what the bookie is up to. 

There are also variations of expected accuracy argu- 
ments for conditioning: relative to her initial credences, 
she will maximize her credences’ expected accuracy upon 
learning whether E iff she conditionalizes (Easwaran, 
2013; Greaves & Wallace, 2006; Leitgeb & Pettigrew, 
2010b; Pettigrew, 2016; Schoenfield, 2017). 

A concern about the accuracy arguments for condi- 
tioning is that they don’t truly show that an agent who 
does otherwise is irrational. The credences C that recom- 
mend conditioning are those the agent accepted before 
she knew whether E, and the conditioning strategy rec- 
ommends C only before it is settled whether E. Once she 
has learned whether E, she recognizes that C is outdated. 
Why should she update in a way that’s recommended by 
credences that are then known to be defective (Pettigrew, 
2016)? Moreover, as long as she adopts any coherent cre- 
dences after learning whether E, any strictly proper inac- 
curacy measure will underwrite the result that the agent 
expects her current credences to be more accurate than 
any alternative credences (Easwaran, 2013). Pettigrew 
(2016) thus contends that these types of arguments can 
only show that rational agents must plan to conditional- 
ize but not that they must follow through. 

A different justification for conditioning comes 
from “minimal mutilation” considerations. Start with a 
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credence function C, and impose the constraint that E is 
to be assigned probability 1 (and that it is the strongest 
such proposition). We want to move to the probability 
function closest to C that meets the constraint. There 
are various “distance” measures on the space of prob- 
ability functions. A notable one is the Kullback-Leibler 
divergence, or relative entropy. For discrete distributions 
P = (pi, p2,...) and Q = (M1, qz, . . .), it is defined as 


D(P, Q) = DP log, (2 


It can be shown that C( -|£) is the function that mini- 
mizes entropy relative to C among all functions that 
meet the constraint. Other distance measures that under- 
write conditioning in this way are the Hellinger distance 
and the variation distance (see Diaconis & Zabell, 1982). 
These results support conditioning insofar as they show 
that an agent who follows it changes her credences con- 
servatively: she doesn’t move away from her original cre- 
dences any more than the evidence warrants, in a certain 
sense. However, one may ask why these particular dis- 
tance measures are epistemically significant, since other 
distance measures deliver results that can disagree with 
conditioning. Moreover, one may ask why it’s important 
to stay close to one’s previous credences when they have 
become outdated. 


5.2 Jeffrey Conditioning 

One limitation of conditioning is that it applies only 
in cases where a thinker learns a piece of evidence with 
certainty. But sometimes we gather information without 
becoming certain of it. For example, suppose you glance 
at a tablecloth in poor light. Based on what you saw, you 
become more confident that the fabric is blue but not 
100% confident. In this type of case, it is controversial 
whether there is any proposition that you have learned 
with certainty (although see Skyrms, 1987). 

Jeffrey conditionalization, or Jeffrey conditioning, is an 
updating rule (named after Jeffrey, 1965) that can incor- 
porate uncertain learning. Suppose {Ej, ... , E,} is a parti- 
tion—a set of mutually exclusive and jointly exhaustive 
propositions—such that the agent’s experience will 
change her credences in the elements of the partition 
from Coa(E;) to Crew(E;). Then, Jeffrey conditioning is this 
updating rule: 


for any A, Cy,(A) = ¥Cyy(A | E:) Coey (E))- 


In words: one’s new credence in A is the weighted sum 
of one’s old conditional credences Coia(A | Ei), where the 
weights are one’s new credences in the partition mem- 
bers. When the thinker becomes certain of a particular E;, 
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Jeffrey conditioning reduces to standard conditioning. 
One interesting difference between standard and Jeffrey 
conditioning is that the former is commutative, while 
the latter is not. This means that when one receives more 
than one piece of evidence, standard conditioning always 
delivers the same credences in the end, regardless of the 
ordering of the updates. By contrast, with Jeffrey con- 
ditioning, the resulting credences need not be the same 
if the updates are made in a different order (Diaconis & 
Zabell, 1982; Levi, 1967). 

There is currently no accuracy-based argument in favor 
of Jeffrey conditioning, but there is a diachronic Dutch 
book argument for it (Armendt, 1980; Skyrms, 1987). 
Additionally, it can be shown that the credence function 
that is arrived at by Jeffrey conditioning is the credence 
function closest to the agent’s original credences that satis- 
fies the constraint of assigning the values Chew(E;) to the 
members of the given partition, according to various 
“distance” measures (Kullback-Leibler, Hellinger, varia- 
tion distance; see Diaconis & Zabell, 1982). 


5.3 Relative Entropy, and More General Dynamics 
The general problem for probability dynamics is this. 
Start with a credence function C, and impose a con- 
straint K; to which credence function meeting K should 
C be revised? We have just seen solutions to this prob- 
lem for the particular constraints that E be assigned cre- 
dence 1 (conditioning) and that specified credences be 
assigned across a partition (Jeffrey conditioning). But we 
can imagine various other such constraints—for exam- 
ple, assign specified conditional credences across a parti- 
tion or assign a probability in the interval [x, y] to E. Any 
“distance” measure gives us a candidate for answering 
this general problem: move to the function closest to 
C, by that measure’s lights, that satisfies K (when there 
is one). Relative entropy is an especially popular such 
measure (although van Fraassen [1981] raises his “Judy 
Benjamin problem” against it for a conditional-credence 
constraint). 


6. Uniqueness 


The putative norms of synchronic and diachronic ratio- 
nality that we have specified so far don’t generally single 
out a unique rational credence function for an agent 
at a time. Some philosophers, usually called permissiv- 
ists, think that for any given body of evidence, there is 
more than one overall doxastic state a rational thinker 
could adopt on its basis (see, e.g., Douven, 2009; Kelly, 
2013; Meacham, 2014; Schoenfield, 2014; for further refer- 
ences, see the excellent survey of the debate in Kopec & 
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Titelbaum, 2016). Proponents of Uniqueness disagree. 
Proponents of Intrapersonal Uniqueness argue that, for 
any given thinker who possesses a particular body of evi- 
dence, there is only one rational overall doxastic state 
this thinker can adopt in response to her evidence. Pro- 
ponents of interpersonal uniqueness argue for the even 
stronger thesis that there is only one rational response to 
any given body of evidence, the same for every thinker. 
(Early defenses of versions of Uniqueness can be found 
in Christensen, 2007; Feldman, 2007; White, 2005; and, 
more recently, in Dogramaci & Horowitz, 2016; Greco & 
Hedden, 2016.*) However, proponents of Uniqueness 
must admit that even if they are right, we probably 
won't ever know what these uniquely rational doxastic 
attitudes are. 


7. Relaxing the Idealizations 


The Bayesian theory of rational credences is highly ideal- 
ized both in the manner in which it represents credences 
and in the normative demands it makes on them. Though 
we have so far granted that philosophers are mainly con- 
cerned with characterizing norms of ideal rationality, we 
may legitimately ask how that project relates to norms of 
rationality for non-ideal agents like us. 


7.1 Imprecise Probabilities 

One way in which the Bayesian view is idealized is in 
its representation of credences as being precisely point- 
valued. But realistically, many of our credences are not 
nearly so specific, and hence representing them this 
way is a distorting simplification of reality. Moreover, it 
has been argued that when our evidence is very unspe- 
cific, it would in fact be unfitting to form a precise cre- 
dence on its basis (see, e.g., Joyce, 2005; Levi, 1985). 
For example, what is your credence that it’s cloudy in 
Vladivostok? It seems hard to come up with a precise 
value. In response, it has been suggested that we represent 
credences as interval-valued. For example, one’s credence 
that it’s cloudy in Vladivostok might be represented as 
[0.1, 0.9]. More formally, and more generally, the entirety 
of someone’s potentially imprecise credences could be rep- 
resented as a set of precise credence functions that together 
capture the person’s imprecise epistemic state. 

However, this approach still has the problem that the 
intervals have artificially precise endpoints: where previ- 
ously there was one precise value, now there are two! 
Furthermore, once people’s credences are represented as 
sets of probability functions, the Bayesian norms of syn- 
chronic and diachronic rationality, and the arguments 
supporting them, have to be reformulated accordingly, 
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which is not always straightforward. There is also a sub- 
stantial debate about how decision theory should work 
when we allow imprecise credences, with Elga (2010) 
raising a difficulty (see Rinard, 2015, for a response). 


7.2 Forgetting/Information Loss 

Another highly idealized assumption made by Bayes- 
ian representations of people’s credences is that think- 
ers never forget or lose information. When an agent 
learns a piece of evidence with certainty, standard Bayes- 
ian models require that she remains certain of it. This 
is of course psychologically highly unrealistic, and it is 
also not obvious that forgetting or losing information 
is a rational defect. Titelbaum (2012) offers an account 
of how standard Bayesian updating rules need to be 
adapted in order to accommodate information loss (for 
important examples, see also Arntzenius, 2003) and self- 
locating credences (concerning who one is, where one is, 
and what time it is). 


7.3 Human Limitations in Approximating 

Ideal Rationality 

Even if we modify the standard Bayesian theory to 
allow for imprecise credences and information loss, the 
requirements of ideal rationality are still so demanding 
that humans cannot be expected to fully comply with 
them. Most obviously, the requirement that one must 
assign every tautology credence 1 exceeds the reason- 
ing capacities of humans. Psychologists have also found 
evidence for other deviations from Bayesian rational- 
ity, such as our tendency not to change our credences 
enough in response to incoming evidence (Slovic & 
Lichtenstein, 1971; see also chapter 8.3 by Gléckner and 
chapter 8.5 by Hertwig & Kozyreva, both in this hand- 
book). There is a major debate in psychology about how 
to interpret the relevant data and about whether such 
results demonstrate that humans are by and large ratio- 
nal (because they often come close to complying with 
Bayesian norms) or irrational (because they standardly 
deviate from them). 

Once we grant that full compliance with Bayesian norms 
is excessively difficult for us to achieve, we may regard the 
norms as regulative ideals that we should approximate as 
much as is feasible. This raises two questions: First, what 
exactly does it mean to approximate ideal Bayesian ratio- 
nality? Second, why is it beneficial to become less irratio- 
nal, even if we can never reach the ideal? 

A good way to measure how well someone’s irrational 
credences approximate ideal rationality is to treat a cre- 
dence function as a vector and to measure its distance 
from the closest vector that represents a rational credence 
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function. For example, suppose someone has the follow- 
ing incoherent credences: C(p) = 0.3, C(~p) = 0.4, in vec- 
tor notation: (0.3, 0.4). Different distance measures will 
identify different credence functions as closest. For exam- 
ple, we may use squared Euclidean distance as our distance 
measure, which is defined for vectors X = (x1, ... ,Xn) and 
Y=(M1,---+))n) as 
A(X,Y) = (x1 - yi) + (X22)? t+ (Xn Yn)’. 


Then the closest coherent credence function is 
C*(p) = 0.45, C*(~p) = 0.55, and the distance between C 
and C* is 0.045. But if we use other measures, such as 
absolute distance or the Kullback-Leibler divergence, we’ll 
get different results (e.g., for the latter, the closest coher- 
ent credence function is C*(p) = 0.438, C*(~p) = 0.572). 
Which distance measure we use becomes important when 
we wonder why it is beneficial for thinkers to become 
more coherent, even if they can’t become perfectly coher- 
ent. Suppose a thinker reduces the incoherence in her 
credence function by replacing her original incoherent 
credence function C, with a less incoherent credence func- 
tion C; that is on the direct path toward the closest coher- 
ent credence function, as measured by squared Euclidean 
distance. C2 is better than C; in at least two respects: C2 is 
more accurate than C; in every possible world, according 
to the Brier score. Moreover, C2 is vulnerable to a lower 
guaranteed loss from a Dutch book than C;, provided 
we standardize the size of the bets the bookie is allowed 
to use. But what if we had used a different measure of 
closeness to coherence? For example, for the Kullback— 
Leibler measure, the accuracy result just mentioned holds 
only if we adopt the log-inaccuracy measure but not if 
we use the Brier score (see De Bona & Staffel, 2017, 2018; 
for earlier approaches to the problem, see also Schervish, 
Seidenfeld, & Kadane, 2002; Zynda, 1996). Hence, we can 
explain why it is good for thinkers to become less inco- 
herent in specific ways, even if they can’t ever have fully 
coherent credences: it gets them a greater portion of the 
accuracy benefits and the Dutch book-avoidance benefits 
that come with perfect coherence. However, these results 
crucially depend on finding the right combinations of dis- 
tance measures, Dutch book, and inaccuracy measures. 


7.4 Disagreement and Higher-Order Evidence 

The fact that we can learn about our own imperfections 
also raises the question of how we should respond to 
information about our own ability to correctly assess 
evidence. Cases in which we disagree with others can 
provide us with such higher-order evidence. Suppose 
you and I have agreed to evenly split our lunch bill. 
We both do the calculations but get different answers. 
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One (or both) of us must have made a mistake (even 
by permissivist lights). What’s the best repair strategy? 
Conciliationists maintain that we should both lower our 
credence that our respective answers are correct (Chris- 
tensen, 2007). Steadfasters, by contrast, claim that if I’ve 
in fact assessed my evidence correctly, higher-order evi- 
dence of this type should not move me (Kelly, 2005).’ Of 
course, not every disagreement indicates that someone 
has made a mistake. Different scenarios require different 
kinds of updates: if you disagree with me because you 
have more evidence, I should defer to you, but not if 
you have less evidence. If we disagree because we started 
from different permissible prior probabilities—before 
the evidence came in—perhaps we can both stick to our 
guns. If we find out we independently arrived at the same 
(high) degree of confidence in something, this might 
warrant raising our confidence even more (Lasonen- 
Aarnio, 2013). Claims about how rational agents should 
respond to disagreement and higher-order evidence thus 
yield further constraints on which prior probabilities are 
rationally permissible. 


8. Conclusion 


Formal epistemology has been one of the greatest growth 
areas in philosophy in the past couple of decades, and 
Bayesian approaches have been among the most domi- 
nant in formal epistemology. A very simple axiomatiza- 
tion codifies the putative synchronic norms on credences, 
and simple updating rules codify the putative diachronic 
norms on them. The result is an elegant theory that 
allows rigorous proofs of various claims that are taken to 
have epistemic significance. No wonder it has become 
so popular! 

Moreover, there is still plenty more work to be done. 
Table 4.1.1 roughly represents the current state of play in 
Bayesian epistemology. The cells with checkmarks rep- 
resent the putative norms that have already been given 
a particular type of putative justification (as far as we 
are aware at the time of writing). An empty cell can be 
empty for one of two reasons: either the missing justi- 
fication is impossible, or it is just a matter of historical 
contingency that nobody has yet proved the requisite 
theorem; indeed, in some cases, perhaps nobody has 
even tried. The corresponding cells may thus be ripe for 
the taking. Moreover, when additional norms and justi- 
fication strategies are proposed, the table itself will grow. 

Then there are various other state-of-the-art topics that 
we have only been able to discuss briefly here: Unique- 
ness; whether credences may, or even must, be imprecise; 
the psychological reality (or otherwise) of Bayesianism 
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Table 4.1.1 
Summary of putative justifications of norms 
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Putative justifications = 


Putative norms | Dutch book Accuracy 


Minimal 
distance 


Maximal 
entropy 


Representation 
theorem 


Probabilism Vv Vv 
Regularity Vv 
Principal Principle Vv 
Reflection Principle Vv 


Indifference Principle 


S88 


S 


Conditionalization 


Jeffrey 
Conditionalization 


Vv 


and the extent to which humans are answerable to its 
norms; disagreement; and so on. And finally, there are 
still other “hot” topics that we have not even touched 
on: other putative norms; the relationship between 
quantitative credences and binary beliefs (which are a 
stock-in-trade of traditional epistemology); the relation- 
ship between credences and knowledge (likewise); quan- 
titative versus comparative credences; Bayesianism versus 
other representations of uncertainty (Dempster-Shafer 
functions, ranking functions, belief revision theory, pos- 
sibility theory, etc.); Bayesian decision theory—or rather, 
rival formulations of it; and more. We may thus be highly 
confident that Bayesian epistemology will remain a fer- 
tile research program for a long time to come! 
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Notes 


1. It’s an interesting question how we might go from the kinds 
of intuitive (nonnumerical) attributions of confidence with 
which we began to numerical degrees of belief. See, for example, 
Fishburn (1986) for discussion of the relationship between com- 
parative and numerical probabilities. Stefansson (2017) argues 
that comparative probabilities are primitive and psychologically 
real but that numerical probabilities are neither. 


2. For alternative axiomatizations that take conditional prob- 
ability to be primitive, see Popper (1959) and Rényi (1970). 


3. Kolmogorov goes on to give an infinitary extension of the 
Finite Additivity axiom: 


3’. (Countable Additivity) If {Aj} is a countably infinite col- 

lection of (pairwise) disjoint events, then 

P U A,] = EP(A,). 

n=1 n=1 

However, the status of this axiom is more controversial (see, 
e.g., de Finetti, 1974), and we will not assume it in what 
follows. Moreover, a sentential reformulation of Countable 
Additivity requires some finessing, as classical logic admits 
only finite disjunctions. 


4. It is a difficult question how conditional probabilities are 
related to probabilities of conditionals. We refer interested 
readers to chapter 4.4 by Pfeifer, chapter 4.6 by Oberauer and 
Pessach, and chapter 6.2 by Over and Cruz (all in this hand- 
book), as well as to Hajek and Hall (1994) and Pfeifer and 
Douven (2014). 


5. A critical discussion of another putative argument for proba- 
bilism, Cox’s theorem, can be found in chapter 4.7 by Dubois 
and Prade (in this handbook). 


6. There are unboundedly many references to both of these on 
the Internet. See also chapter 8.2 by Peterson (this handbook). 


7. To be sure, the original Principal Principle holds as a con- 
straint on rational agents more generally, assuming that they 
update by conditioning on evidence that is limited to the 
past and present—more on conditioning shortly. However, as 
we will see, the main arguments for conditioning have been 
questioned, as has conditioning itself. Lewis (1994) also offers a 
“New Principle” that explicitly supplants the original principle. 


8. Carnap (1950) attempts to characterize a unique logical 
probability function and can thus also be seen as an early 
defender of Uniqueness. Various proponents of objective 
Bayesianism also fall into this camp (e.g., L. Williamson, 2010). 


9. The literature on disagreement and higher-order evidence 
is vast. Matheson (2015) provides an overview of the debate. 
See also recent anthologies on the topic, such as Feldman and 
Warfield (2010) and Christensen and Lackey (2016). 
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4.2 Bayes Nets and Rationality 


Stephan Hartmann 


Summary 


Bayes nets are a powerful tool for researchers in statis- 
tics and artificial intelligence. This chapter demonstrates 
that they are also of much use for philosophers and psy- 
chologists interested in (Bayesian) rationality. To do so, 
we outline the general methodology of Bayes nets mod- 
eling in rationality research and illustrate it with several 
examples from the philosophy and psychology of rea- 
soning and argumentation. Along the way, we discuss 
the normative foundations of Bayes nets modeling and 
address some of the methodological problems it raises. 


1. The Virtues of Bayes Nets 


Bayes nets (or “Bayesian networks”) are a powerful tool 
for researchers in statistics and artificial intelligence. The 
reason for this consists in the many scientific virtues of 
Bayes nets. One of these virtues is their representational 
power: Bayes nets represent in an intuitive way the con- 
ditional independencies that hold between variables. 
Exploiting these conditional independencies, Bayes nets 
allow for a compact representation of a joint probabil- 
ity distribution over n variables: what would ordinarily, 
for binary variables without any additional constraints, 
require the specification of 2"- 1 parameters here needs 
significantly less. This has enormous practical advan- 
tages and hence initiated the development of probabi- 
listic expert systems and other technical applications. 
Another virtue of Bayes nets is their algorithmic power: 
Bayes nets come with efficient algorithms to derive from 
the joint distribution whatever marginal or conditional 
probability one is interested in. Last but not least, the 
theory of Bayes nets is very elegant, and not much is 
needed to successfully apply it to new problems. 

These are some of the reasons why Bayes nets have 
already found so many applications in various parts of 
science and engineering. The goal of this chapter is to 
demonstrate that Bayes nets are also of much use for 
philosophers and psychologists in the field of (Bayesian) 


rationality. We will see that Bayes nets can be naturally 
integrated into the Bayesian framework and help solving 
problems that would otherwise be hard to address. In 
the following pages, we will outline the general meth- 
odology of Bayes nets modeling in rationality research 
and elaborate on the various functions of these models. 
The methodology will then be illustrated by analyzing a 
number of problems and questions from the philosophy 
and psychology of reasoning and argumentation. 

The remainder of this chapter is organized as follows: 
section 2 provides a concise introduction to the theory 
of Bayes nets. Section 3 introduces Bayesian rational- 
ity. Here we distinguish between the general Bayesian 
framework and the models that are constructed within 
the framework to address a specific problem or question 
from rationality research. Bayes nets play a role in the 
latter but not in the former. We illustrate the method- 
ology by examining a number of increasingly complex 
confirmation scenarios. Section 4 considers two exam- 
ples from the philosophy and psychology of reasoning 
and argumentation in more detail. We will see that Bayes 
nets models help the rationality researcher to recon- 
struct certain reasoning and argumentation schemes 
and to identify possible holes in an argument (and to 
suggest a remedy). The section closes with the sketch of 
a general theory of Bayesian argumentation with a spe- 
cial focus on the role of indicative conditionals. It builds 
heavily on the use of Bayes nets. Section 5, finally, closes 
with a short outlook. 


2. Bayes Nets in a Nutshell 


A Bayes net organizes a set of variables into a directed 
acyclic graph (DAG). A DAG is a set of nodes and a set 
of arrows between those nodes. The only constraint is 
that there are no closed paths formed by following the 
arrows. A “root node” is a node with outgoing arrows 
only, a “parent” of a given node is a node from which 
an arrow points at the given node, and a “descendant” 
of a node is one that is pointed at by a corresponding 
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arrow. Each node represents a propositional variable, 
which can take any number of mutually exclusive and 
jointly exhaustive values. To make a DAG into a Bayes 
net, one more step is required: we need to specify the 
prior probabilities for the variables in the root nodes and 
the conditional probabilities for the variables in all other 
nodes, given any combination of values of the variables 
in their respective parent nodes. 

The arrows in a Bayes net carry information about the 
conditional independence relations between the vari- 
ables in the DAG. This information is expressed by the 
Parental Markov Condition (PMC): 


PMC _ A variable represented by a node in a Bayes net is 
independent of all variables represented by its non- 
descendant nodes, conditional on all variables repre- 
sented by its parents. 


Here is an illustration: consider the Bayes net in 
figure 4.2.1, involving the three propositional variables 
A, B, and C. Node C is a root node. It is the parent of 
A and B, and A and B are the children of C. Applying 
PMC, we find that A  B| C (read: A is independent of B 
given C). One also says that C screens off A from B. 

Compare this Bayes net with the one in figure 4.2.2. 
Here B is the parent of C and C is the parent of A. Node 
A is a child of C and at the same time a descendant of B. 
Interestingly, applying PMC, we find that A 1 B|C also. 
The networks in figures 4.2.1 and 4.2.2 therefore repre- 
sent the same conditional independence structure. That 
is, if a modeler has reason to assume that A ll B| C, then 
this conditional independence can be represented in a 
Bayes net in two ways. To single out one of them, further 
information is needed. 


Figure 4.2.1 
The “common cause” network. 


Figure 4.2.2 
The chain network. 
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Figure 4.2.3 
The collider (or “common effect”) network. 


Finally, let us consider the Bayes net in figure 4.2.3. 
Here the variables A and B are the parents of C. Apply- 
ing PMC, we find that A 1 B, that is, A is unconditionally 
independent of B (one could also write A . B| @, where © 
represents the empty set). Hence, the conditional inde- 
pendence structure instantiated in the Bayes net in fig- 
ure 4.2.3 differs from the one in figures 4.2.1 and 4.2.2. 

Conditional independence structures can be investi- 
gated in general, and there is a rich literature on the topic. 
Most important, conditional independence structures 
(i.e., the three-place relation: .-|-) satisfy the so-called 
semi-graphoid axioms (A. P. Dawid, 1979; Spohn, 1980), 
which can be used to derive new conditional indepen- 
dencies from already known ones.' This is important as 
it is easy to see that PMC does not allow us to identify 
all conditional independencies that hold in a DAG. Con- 
sider the Bayes net in figure 4.2.2. Here PMC implies only 
that A . B|C. However, it also seems to be the case that 
B i A|C, which does not follow from PMC. This condi- 
tional independence follows from A i. B|C and the sym- 
metry axiom (which is one of the semi-graphoid axioms). 
Applying the semi-graphoid axioms to find all conditional 
independencies is rather cumbersome, and so it would be 
helpful to have one criterion that identifies all conditional 
independencies in a DAG in a straightforward way. This 
is the d-separation criterion, which is explained in text- 
books such as Darwiche (2014) and Pearl (1988). 

While conditional independence structures can be 
studied abstractly, we are only interested in probabilis- 
tic conditional independence structures. For these, it is 
helpful to recall that two propositional variables A and B 
are probabilistically independent with respect to a prob- 
ability measure P if and only if P(A, B) = P(A)P(B) for all 
values of A and B. Equivalently, A and B are probabilisti- 
cally independent with respect to a probability measure 
P if and only if P(A|B) = P(A) for all values of A and B, 
where P(A|B) stands for the conditional probability of 
A given B, which is defined as P(A,B)/P(B) if P(B) > 0.? 
Generalizing this definition, two propositional variables 
A and B are probabilistically independent given the 
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variable C with respect to a probability measure P if and 
only if P(A,B|C) = P(A|C)P(B|C) for all values of A, B, 
and C. It is easy to see that this definition is equivalent 
to the following one: A and B are probabilistically inde- 
pendent given C with respect to a probability measure 
P iff P(A|B,C) = P(A|C) for all values of A, B, and C. That 
is, once the value of C is known, learning the value of B 
does not change the probability of A (provided that all 
conditional probabilities are defined). 

While conditional independencies can be read off 
from the network structure, a probability distribution 
defined over the DAG is needed to make specific proba- 
bilistic inferences. To do so, we express the joint prob- 
ability distribution over a set of variables Ai,..., An, 
which is organized into a DAG in terms of the prior 
probabilities of all root nodes and the conditional prob- 
abilities of all child nodes, given any combination of 
values of the variables in their respective parent nodes. 
Let pa(A;) denote the set of parents of A;. Then an appli- 
cation of the chain rule of the probability calculus 
yields the following expression (“the product rule”) for 
the joint probability distribution over all variables:* 


P(A,,...,A,) = [[24 pac) 


= P(A,| pa(A;)) P(A; | pa(A,)) + + + P(A, |pa(A,))- 


(1) 


Let us illustrate the application of equation 1 with 
the DAG in figure 4.2.3. To make the DAG a Bayes net, 
we assume that all variables are binary (with their val- 
ues represented by A, 7A, etc.) and specify the (uncon- 
ditional) probabilities of all root nodes, that is, P(A) = a 
and P(B) = b, and the conditional probabilities of the 
child node given the four combinations of values of the 
two variables in its respective parent nodes, that is, 


P(C|A,B) = a, 
P(C| 7A,B) = y, 


P(C|A, 7B) = B, 
P(C|7A,7B) = 6. 


The product rule (ie., equation 1) then allows us to 
compute whatever marginal or conditional probability 
we are interested in. For example, one easily sees that 
P(A, B, C) = P(A)P(B)P(C|A, B) = aba and that P(A, B, C) = 
P(A)P(AB)P(C|A, 4B) =abB, where we have used the 
shorthand notation x := 1—x, which we will also use 
below. 

Similarly, one can calculate P(A| B,C) and P(A|C) 
and show that the two probabilities are not identical. 
That is, A and B are unconditionally independent but 
conditionally dependent, given C. Fixing the value of 
the “common effect” variable C renders the “causes” A 
and B dependent.* 
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3. Bayesian Rationality 


Bayesianism is the leading theory of uncertain rea- 
soning. Its starting point is the psychological truism 
that people believe contingent propositions such as 
“It will rain tomorrow” more or less strongly: they 
assign a certain degree of belief to a proposition. But 
what are rational degrees of belief? How can they 
be combined? And how should one change them if 
new evidence becomes available? To address these 
questions, we need a calculus for the representation 
of degrees of belief (i.e., a theory about the statics of 
rational belief), rules for changing them (i.e., a theory 
about the dynamics of rational belief), and a norma- 
tive foundation for both. 

Before moving on, it is useful to distinguish between 
the Bayesian framework and the models that are con- 
structed within this framework. While the framework 
lays out the general features of Bayesian rationality and 
comes with a normative foundation, the models represent 
specific reasoning situations and help the researcher to 
tackle concrete problems. These models often involve 
Bayes nets. 


3.1 The Bayesian Framework 

Let us begin with the static part of Bayesianism. Here 
Bayesians identify degrees of belief with probabilities. 
As a consequence, the probability calculus puts specific 
constraints on the degrees of belief of an agent. For 
instance, if a rational agent assigns a degree of belief of 
.3 to the proposition “It will rain tomorrow,” then this 
agent has to assign a degree of belief of .7 to the proposi- 
tion “Tt will not rain tomorrow,” as the latter proposition 
is the logical negation of the former and the probabil- 
ity of a proposition and its negation sum up to 1. More 
generally, a probability distribution P is defined over a 
Boolean algebra & of propositions, which comes with 
rules for the combination (a and v) and negation (7) of 
propositions. In a first step, the agent fixes the algebra, 
that is, she identifies all relevant propositions.° In the 
second step, the agent specifies a joint probability distri- 
bution over the algebra &. As a result, the beliefs of the 
agent are coherent. 

Turning to the dynamic part, Bayesians specify rules 
for changing (“updating”) probabilities once new infor- 
mation becomes available. Let us assume, for example, 
that an agent has partial beliefs about the propositions 
A, B, and C. They are represented by propositional vari- 
ables A, B, and C, and a prior probability distribution P 
is defined over them. The agent then learns that A is the 
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case. That is, the new probability of A, that is, P’(A), is 1. 
Here P’ denotes the new (“posterior”) probability distri- 
bution of the agent. So far, we only know the new value 
of the probability of A. But what are, for example, the 
new probabilities of B and C? And what is the full new 
joint probability distribution over A, B, and C? Bayesians 
argue that in this case, the agent should update her prob- 
ability distribution according to the principle of Condi- 
tionalization (“Bayes’ theorem”), that is, she should set 


P'(X) = P(X|A) 


for any proposition X in the algebra & under consideration. 

But why should we identify degrees of belief with 
probabilities? And why should one update according to 
Conditionalization? That is, what is the normative foun- 
dation of the Bayesian framework? There are different 
ways to provide such a normative foundation, and it is 
a strength of Bayesianism that it is supported by a wide 
variety of such arguments. The two most popular types of 
arguments are pragmatic (“Dutch book arguments”) and 
epistemic (“epistemic utility theory”). They are explained 
in chapter 4.1 by Hajek and Staffel (this handbook). 

It is interesting to note that the dynamic part of 
Bayesianism (i.e., Conditionalization) can also be jus- 
tified in a different way. To do so, we assume that the 
agent wants to be as conservative as possible with regard 
to changing her beliefs. That is, the agent is undogmatic 
and willing to modify her beliefs once new informa- 
tion comes in, but she wants to make sure that overall, 
her beliefs change as little as possible. This seems to be 
psychologically plausible and is also part of other theo- 
ries of belief revision such as the AGM model, which is 
explained in chapter 5.2 by Rott (this handbook). More 
specifically, the principle of Conservativity demands that 
an agent who learns a new item of information make 
sure that the new probability distribution P’ takes this 
new information into account as a constraint but also 
requests that P’ differ as little as possible from the old 
(prior) probability distribution P. 

Making this idea precise requires the specification of 
a measure of the distance between two probability dis- 
tributions. It turns out that the most interesting and 
useful measures do not satisfy the axioms of a math- 
ematical distance (i.e., of a metric space). For instance, 
the Kullback-Leibler divergence is not symmetrical and 
violates the triangle inequality. However, minimizing 
the Kullback-Leibler divergence yields Conditionaliza- 
tion if one takes into account the constraint that the 
probability of some proposition in the algebra shifts to 
1 (see Diaconis & Zabell, 1982; Eva, Hartmann, & Rafiee 
Rad, 2020). 
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It is instructive to note an analogy to Newtonian 
mechanics here. Newtonian mechanics, too, provides 
a modeling framework. It specifies a static part (mass 
points, etc.) and a dynamic part (Newton’s second law 
as a general dynamical law). Furthermore, there are jus- 
tifications for both parts. What is more, the Newtonian 
framework comes with various assessment criteria and 
a (perhaps somewhat implicit) methodology for model 
construction (see Giere, 1990). This also holds for Bayes- 
ian modeling, which is the topic of the following sub- 
section. Before that, however, we introduce Bayesian 
Confirmation Theory (BCT), which is the central phil- 
osophical application of Bayesianism. Its most general 
aspects are part of the Bayesian framework. 

While qualitative confirmation theories formulate 
criteria that inform us whether or not a piece of evidence 
E confirms a hypothesis H (Sprenger, 2011), quantitative 
theories of confirmation (such as BCT) also tell us how 
much E confirms H. According to BCT, an agent starts 
with a subjective degree of belief that a certain hypothe- 
sis H is true—the prior probability P(A) of the hypothesis. In 
the next step, a more or less expected piece of evidence, E, 
comes in. While E was uncertain before, it now becomes 
certain. To make sure that the beliefs of the agent remain 
coherent, the agent updates the probability of H and 
assigns a posterior probability P'(H) to the hypothesis using 
Conditionalization, that is, P’(H) = P(H|E). This can also 
be expressed as 


P(A) 


7s P(H) + P(=H)x’ 


with the likelihoods p := P(E| H) and q := P(E| +H) and the 
likelihood ratio x := q/p. Now E confirms H iff the poste- 
rior probability P’(H) (after learning E) is greater than 
the prior probability P(H). Evidence E disconfirms (or 
“falsifies”) H iff the posterior is smaller than the prior. 
If P’(H) = P(H), then E is irrelevant for H. Equivalently, 
E confirms H iff the likelihood ratio x < 1,E disconfirms 
H iff x>1, and E is irrelevant for H iff x = 1. For more 
on BCT, see Crupi (2016), Huber (2007), and chapter 4.3 
by Merin (this handbook), which also discusses how to 
measure evidential relevance. 


3.2 Bayesian Models 

The general framework just described cannot be applied 
directly to concrete problems and questions. To do so, 
the agent has to specify (1) the relevant variables and 
(2) their relations. This may involve a considerable amount 
of modeling, as it isn’t always clear which variables are 
the relevant ones.’ Besides, different methodological val- 
ues may conflict. One may, for example, favor a rather 
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simple, intuitive, and understandable model. This can 
be achieved by taking into account only a small number 
of variables. Sometimes it is also possible to effectively 
combine various variables into one macro-variable. On 
the other hand, one might also want to account for the 
details of a reasoning scenario, and additional proposi- 
tions might be needed for this. The modeler has to decide 
what to do. The same holds for the assumed relations 
between the variables. Are two variables really strictly 
(conditionally) independent? This may be controversial, 
and strict independencies are hard to come by. 

However, once the modeler has decided what the rel- 
evant variables and their relations are, Bayes nets come 
in and help with the representation and the computa- 
tions.* We will see that certain independence assump- 
tions can do a lot of work and, if properly motivated, 
reduce the amount of subjectivity that one might 
complain about in a Bayesian model, due to the con- 
siderable freedom one otherwise has to fix a prior prob- 
ability distribution. The structural constraints imposed 
in a Bayes net alone already contribute a great deal to 
the solution of a problem, and in the study of Bayesian 
rationality, probabilistic models often have Bayes nets 
as integral parts. Besides, once the Bayes net is fixed, the 
problem or question that prompted the construction of 
the model can be addressed by powerful mathematical 
machinery. 

It has long been noted that models are an indispens- 
able part of science.” They have important (pragmatic 
and epistemic) functions in the research process, and 
this also holds for the Bayes net models we are consider- 
ing in this chapter. Here is a (nonexhaustive) list of the 
functions of Bayes net models in rationality research: 


* Models help to apply the (Bayesian) theory. We men- 
tioned already that not much follows from the gen- 
eral framework for a particular problem or question; 
it requires the model and the specification of details. 


* Models help to test the (Bayesian) theory. Once the 
model is constructed, its consequences can be con- 
fronted with empirical data. Note, however, that 
Bayes net models are normative models, which raises 
additional problems (see Colyvan, 2013; Titelbaum, 
2021). 


* Models help to solve concrete problems, as many exam- 
ples show (see also the illustrations in figure 4.2.4). 

* Models accommodate and explain experimental data. 

* Models help the researcher to reconstruct and analyze 
various reasoning and argumentation schemes and 
check their rationality. 
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* Bayes net models provide a compact and intuitive repre- 
sentation of a reasoning and argumentation situation 
and help with the bookkeeping of the variables and 
their relations. 


3.3. Modeling Confirmation Scenarios 

Large parts of the literature on BCT are only concerned 
with scenarios involving two variables—one represent- 
ing the hypothesis (H) and the other representing the 
evidence (E). However, while much can be learned from 
focusing on simple scenarios (see, e.g., Earman, 1992; 
Howson & Urbach, 2006), it is clear that actual confir- 
mation scenarios are much more complex and may raise 
new and intricate issues that do not show up in scenarios 
involving only two variables. If we want to reconstruct 
and analyze more complex confirmation scenarios, then 
Bayes nets prove to be extremely helpful. Without the 
use of Bayes nets, it is hard to keep track of the vari- 
ous dependencies and independencies and to properly 
evaluate what is going on, as the following example 
illustrates. 

We consider the testing of a hypothesis with partially reli- 
able measurement instruments. To begin with, we consider 
a situation where the evidence is uttered by a partially 
reliable information source. This information source could 
be a measurement instrument (which outputs a binary 
result, e.g., a positive or negative X-ray) or the testimony 
of a witness (e.g., in a murder case), again modeled as a 
binary variable, such as “I saw the suspect on the crime 
scene” or “I didn’t see the suspect.” 

Such scenarios can be modeled by fixing the rates of 
false positives and false negatives of the information 
source. The rate of false positives (f,) measures how often 
an instrument outputs “true” when in fact the hypothesis 
is false. Similarly, the rate of false negatives (f;) measures 
how often an instrument outputs “false” when in fact the 
hypothesis is true. Ideally, both rates are zero, but in real- 
istic cases, this is almost never the case. To proceed, it is 
easy to see that f; and f; are related to the likelihoods men- 
tioned above: f, = P(E| -H) = q and f, = P(-E|H) =1-p. 
While these likelihoods (or rates) are often available (e.g., 
in medical testing), this is not always the case (e.g., for wit- 
ness reports). This suggests that one might want to con- 
struct more complex models for the information-gathering 
process of an agent. 

Here is a simple example. Consider an agent who 
entertains the hypothesis H (“Paul is the murderer”). She 
then receives a witness report Rep (which is the evidence) 
to the effect that Paul indeed is the murderer. The agent 
then assumes—and this is a modeling assumption—that 
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Figure 4.2.4 
A Bayes net representing the test of a hypothesis with a partially 
reliable information source. 


the witness is either reliable or not: if the witness is reli- 
able, then she is fully reliable and therefore always tells 
the truth. That is, if Paul is the murderer, she reports that 
he is the murderer, and if Paul is not the murderer, she 
reports that Paul is not the murderer. However, if she is 
not reliable, then she is a randomizer, that is, she reports 
with a certain probability a (the so-called randomization 
parameter) that Paul is the murderer, independently of 
whether or not Paul is the murderer. Finally, the agent 
assumes that it is uncertain whether or not the witness is 
reliable and assigns a prior probability to the proposition 
Rel (“The witness is reliable”). 

Next we represent this situation with the (“collider”) 
Bayes net in figure 4.2.4 and set P(H) = h, P(Rel) = r, and 


P(Rep|H, Rel) = 1, 
P(Rep|H, 7Rel) = a, 


P(Rep | 7H, Rel) = 0, 
P(Rep| 7H, 7Rel) = a. 


With this, we calculate P'(H) = P(H | Rep), that is, the pos- 

terior probability of the hypothesis after receiving a posi- 

tive witness report: 

_ P(H, Rep) _ Xp P(H, Rel, Rep) 
P(Rep) Lu, ra P(A, Rel, Rep) 

_ Xp PCH) P(Rel) P(Rep |H, Rel) 

~ Dirra PCA) P(Rel)P(Rep |H, Rel) 

_ _ A(r+ar) 

~ h(r+ar)+har 


P(A) 


(2) 


_ h(r+ar) 
~ hr+ar * 


It is interesting to note that this model can be simulated 
by a two-variable model (using the variables H and Rep) 
with the corresponding likelihoods p := P(Rep|H) = r+ar 
and q := P(Rep|7H) = ar. From this, one also sees that 
p > q (for r > 0). Hence, as expected, Rep always confirms 
H. From equation 2, we obtain that P’(H) = P(H) forr = 0. 
Likewise, for r= 1, we find that P’(H) = 1. This makes 
sense: if a perfectly reliable information source tells us 
that H is true, then H is true. 

Because this three-variable model can be simulated by 
a two-variable model, it is not strictly necessary to study 
it in detail. However, the model allows us to reduce the 
likelihoods p and g to some parameters that are easier to 
grasp and interpret (namely, a and r). The model there- 
fore provides (or suggests) a mechanism that generates 
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the likelihoods, but once these likelihoods are known, 
one can proceed with the two-variable model. 

Things get more interesting when one studies more 
complex scenarios. Let us assume, for instance, that the 
agent receives two positive reports from two partially 
reliable information sources. We can then distinguish 
two scenarios and ask which of them provides more 
confirmation for the hypothesis in question. In the first 
scenario, the two reports are independent from each 
other. This is modeled by fixing two root nodes Rel; 
and Rel, (see figure 4.2.5). In the second scenario, we 
assume that the reports are dependent and model this 
by working with only one root node Rel, which is a par- 
ent of both Rep; and Repz (see figure 4.2.6). One would 
expect that the first scenario provides more confirma- 
tion, ceteris paribus, as the two reports are independent. 
Here, the ceteris paribus clause makes sure that the pri- 
ors of H and Rep (and of Rep; and Repz, respectively) 
are the same and that the randomization parameter is 
the same in both scenarios. But is this really the case? 
Do independent positive reports always result in more 
confirmation? A detailed analysis shows that the answer 
to this question is “no”: the second scenario provides 
more confirmation if the values of a and rare sufficiently 
small.'° 

Extending and generalizing these ideas, Landes (2020) 
provides a systematic assessment of the variety-of-evidence 
thesis. This is the claim that more varied evidence con- 
firms more strongly than less varied evidence. Landes 


Figure 4.2.5 
A Bayes net representing the test of a hypothesis with two inde- 
pendent instruments. 


Figure 4.2.6 
A Bayes net representing the test of a hypothesis with two 
dependent instruments. 
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explores in detail under which conditions the thesis 
holds. Further applications of the simple witness model 
described above can be found in the psychological litera- 
ture (see, e.g., Hahn, Merdes, & von Sydow, 2018; Harris, 
Hahn, Madsen, & Hsu, 2016). 

There are many other applications of Bayes nets in BCT. 
For instance, Dardashti, Hartmann, Thébault, and Wins- 
berg (2019) investigate whether so-called analogue simula- 
tions can be used to confirm a theory. The (very rough) idea 
is this: A theory T predicts a phenomenon 4g, but for certain 
(practical) reasons, the prediction cannot be directly tested. 
However, it turns out that there is another theory T’, which 
is in important respects analogous to T and predicts the 
analogous phenomenon gq’. Fortunately, the prediction 
of ¢' can be tested. The observation of gy’ then confirms 
T’, but does it also confirm T (as some authors argue for 
some cases)? Using the machinery of Bayes nets, Dardashti 
et al. (2019) show that this is indeed the case, given that 
certain conditions hold. Further applications concern the 
use of Bayes nets in legal reasoning. For example, Lagnado 
and collaborators (e.g., Connor Desai, Reimers, & Lagnado, 
2016; Fenton, Neil, Yet, & Lagnado, in press) use Bayes nets 
to represent complex legal scenarios, demonstrating the 
power and the flexibility of the approach (see also chap- 
ter 11.4 by Prakken, this handbook). 


4. Bayesian Reasoning and Argumentation 


Confirmation theory and the analysis of confirmation 
scenarios are an important focus of Bayesianism. Bayes- 
ianism provides a clear criterion for when it is rational 
for an agent to claim that a piece of evidence supports 
a given hypothesis, and Bayes nets help to apply this 
theory to concrete cases so that the theory can be put to 
work. At this point, however, it is important to note that 
epistemic rationality comprises more than confirmation 
theory. In many cases of reasoning and argumentation, 
it is not immediately clear that such scenarios can be 
reconstructed (or represented) as confirmation scenar- 
ios. Interestingly, though, often it is possible. We illus- 
trate this point with the sketch of a Bayesian account of 
argumentation (see also chapter 5.5 by Hahn & Collins 
and chapter 5.6 by Woods, both in this handbook). 

The starting point of Bayesian argumentation is the 
observation that the premises and conclusions in typi- 
cal real-life arguments are uncertain to the agent. We are 
more or less convinced of the premises of an argument, 
given the evidence we have for them (and our background 
knowledge), and this uncertainty transfers to the argu- 
ment’s conclusion. This holds whether the underlying 
argument scheme is valid or not. Hahn and Oaksford 
(2007) have observed that even so-called logical fallacies 
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(such as the argument scheme Denying the Antecedent) 
can be powerful in the sense that they can increase an 
agent’s degree of belief in the conclusion of the argument. 
Hahn and Oaksford argue convincingly that it is not only 
the logical structure that is important when it comes to 
assessing the strength of an argument but also the content 
of the premises (see also Hahn & Hornikx, 2016). Gener- 
alizing from these insights, Eva and Hartmann (2018a) 
develop a general theory of Bayesian argumentation 
according to which “Argumentation is learning.” This 
slogan connects argumentation with confirmation, and 
we will sketch the corresponding theory now. Some of 
its applications and further developments are discussed in 
the following subsections. 

We consider an agent (agent 1) who entertains a set of 
propositions with a prior probability distribution P defined 
over it. This can be represented by a Bayes net. Now another 
agent (agent 2) wants to convince agent 1 of some propo- 
sition. She decides to do so in an indirect way by manip- 
ulating the beliefs of agent 1 about the premises of an 
argument. More specifically, agent 2 aims at getting agent 
1 to increase her degree of belief in some of the premises. 
To make sure that her overall degrees of belief are coherent, 
agent 1 then updates on that new information, which leads 
to a new probability of the conclusion. Hence the slogan 
“Argumentation is learning.” If the new probability of the 
conclusion is greater than the old one, then the argument 
has some force; if not, then not. Note that the possible logi- 
cal relationship between the premises and the conclusion 
plays a role in the updating process. This has been noted 
long ago by Suppes (1966) and Adams (1996). 

As an illustration, consider modus ponens, that is, the 
rule 


AC 
A 


C 


and assume that the agent has a prior probability distribu- 
tion P defined over the two variables. To reconstruct the 
argument in Bayesian terms, we assume that the agent 
learns the premises of the argument with certainty. We can 
then use Conditionalization to compute the new probabil- 
ity of C. Representing the conditional AC by the corre- 
sponding material conditional —A v C finally yields 


P(A,C, AVC) _ | 


PC) = P(C|A, aAVO) = eee 


Hence the argument succeeds. As a result, the agent 
increases the probability of the conclusion to 1 and thus 
becomes certain that C is true. In less ideal circumstances, 
for example, if the probability of the minor premise does 
not increase to 1 but to some value smaller than 1, one 
finds that the probability of the conclusion in a modus 
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ponens argument increases but never reaches 1. Interest- 
ingly, for valid arguments, this always holds: if the con- 
ditional is learnt with certainty and the probability of 
the minor premise increases, then the probability of the 
conclusion of the argument increases. This is not the case 
for invalid arguments, which is a reason to prefer valid 
ones (see Adams, 1996). If the argument scheme is invalid, 
then it depends on the prior probability distribution of 
the agent whether or not the probability of the conclusion 
increases. We will come back to this in subsection 4.3. But 
before, let us illustrate Bayesian argumentation with two 
examples that also illustrate the importance of Bayes nets. 


4.1. The No-Alternatives Argument 
Consider the following argument: 


A scientist entertains a theory H that satisfies several desir- 
able conditions. Unfortunately, however, the theory 
cannot be tested empirically because it is mathemati- 
cally too difficult to derive predictions from the theory 
or it is impossible to experimentally test the predic- 
tions of the theory. At this point, the scientist argues 
as follows: “Look, my colleagues and I tried hard to 
find an alternative to H that also satisfies the desirable 
conditions, but despite a lot of effort and brain power, 
we did not succeed. This supports the claim that H is 
true.” That is, the scientist argues that an observation 
about the performance of the scientific community 
can provide a reason in favor of a scientific theory. 


This is the No-Alternatives Argument (NAA). Quite 
recently, it has been put forward by defenders of string 
theory, which is a candidate for the most fundamental 
theory of physics that arguably provides a unified account 
of all four fundamental forces of nature (i.e., gravity, elec- 
tromagnetism, and the weak and strong nuclear forces). 
Unfortunately, string theory lacks direct empirical confir- 
mation, and no one expects this situation to change in 
the foreseeable future. This raises the question why scien- 
tists stick to a theory that is not (and perhaps never will 
be) confirmed by empirical data. 

In his recent book String Theory and the Scientific Method 
(2013), R. Dawid suggests that the NAA is convincing, 
given certain conditions, and R. Dawid, Hartmann, and 
Sprenger (2015) provide a Bayesian analysis of the NAA. 
We present a slightly simplified version of their recon- 
struction and consider the following argument: 
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P,: Theory H satisfies several desirable conditions. 


P,: Despite a lot of effort, the scientific community has 
not yet found an alternative to H that also satisfies 
these conditions. 


C: Hence we have one good reason in favor of H. 


Let F be the proposition “The scientific community has 
not yet found an alternative to H.” One then has to show 
that F confirms H. Using BCT, we therefore have to show 
that P’(H) = P(H|F) > P(A). Note that F is neither a deduc- 
tive nor an inductive consequence of H. Hence there 
cannot be a direct probabilistic dependence between the 
corresponding propositional variables. It is therefore natu- 
ral to look for a third variable that facilitates the depen- 
dence. One possibility is a “common cause” structure with 
the two “effects” H and F and a so-far-unknown “common 
cause” variable Y that screens off H and F from each other. 
Such a structure seems appropriate as it renders H and F 
dependent if the agent does not know the value of Y, and 
H and F become independent once this value is known. 
But what could this variable be? We need to find another 
active variable about which the agent has beliefs and which 
is related to H and F in a simple and easy-to-justify way. 
(We certainly want to avoid that the result depends 
too much on the idiosyncrasies of the specific model.) 
R. Dawid et al. (2015) suggest the introduction of the mul- 
tivalued variable Y, which has the following values: 


Y;: There are i distinct alternative theories that satisfy the 
desirable conditions. 


Here, i runs from 0 to some maximal value N. Each Y; is 
a statement about the number of existing theories that 
satisfy the same conditions as H. It is easy to see that 
Y screens off F from H: Once we know the value of Y, 
learning F does not tell us anything new about the prob- 
ability of H. To assess it, all that matters is that we know 
how many equally suitable candidate theories there are. 
Y facilitates the probabilistic dependence between F and 
H, since if there is only a small number of alternative 
theories, this would provide an explanation for why the 
scientists have not yet found any (i.e., it would explain 
F). In addition, if there are only a few alternative theo- 
ries, this should also probabilistically affect our belief in 
the available theory. We finally arrive at the Bayes net 
depicted in figure 4.2.7 and formulate the following four 
plausible conditions (see unnumbered list below): 


NAA1: VWiz0, yj:= P(Yi) €[0, 1). 

NAA2: hh; := P(H| Y;) are monotonically decreasing in i. 
NAA3:  f,:= P(F| Yj) are monotonically decreasing in i. 
NAA4: 


Hand F are conditionally independent given Y, that is, P(H|F, Y;) = P(H| Yi). 
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Figure 4.2.7 
A Bayes net depicting the No-Alternatives Argument. 


It can then be shown that the following theorem 
holds: 


Theorem 1. Let P be a probability distribution satisfy- 
ing the conditions NAA1-—NAA4. Then F confirms H, 
that is, P(H| F) > P(A), iff there exists a pair (i, j) with 
i>j such that (1) yy, > 0, (2) fi< fi, and (3) hj < hy. 


This theorem allows for an assessment of the NAA. One 
may, for example, question NAAI. Are we really uncer- 
tain about the number of alternatives? Doesn’t the 
underdetermination thesis in philosophy of science tell 
us that the number of alternatives to a given theory is 
always infinite? If this is so, then a rational agent should 
set y.. = 1 and the NAA does not go through. A defender 
of the NAA will therefore have to argue why y.. (and 
all other y;, for that matter) should be assigned values 
smaller than 1. It has to be shown, then, that an agent 
can never be certain about the number of appropriate 
alternatives to a given theory. 

Here is another issue. The NAA relies on how well 
F probes the variable Y. Note that there can be several 
complicating factors. For one, there might be an alter- 
native explanation for why the scientific community 
has not yet found an alternative. For instance, R. Dawid 
et al. (2015) introduce an additional node D represent- 
ing the difficulty of finding an alternative theory. The 
observation of F then may only confirm D (i.e., that it 
is very difficult to find an alternative theory). However, 
D is probabilistically independent of H, and hence the 
observation of F may not confirm the theory in question. 

Note also that the NAA relies on it being possible to 
establish F in the first place. However, it is a nontrivial 
task to find agreement among the members of the scien- 
tific community about the existence or nonexistence of 
alternative theories. And even if there is agreement, this 
is only probative of the number of existing alternatives 
(i.e., the value of the variable Y), provided that the sci- 
entific community has attempted to explore the space of 
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alternative theories and has considered all problems one 
may encounter in this endeavor. 

We conclude that the formal reconstruction of the NAA 
using Bayes nets helps the reasoner to put all assumptions 
on the table and highlights issues that need to be addressed 
to make the argument, if at all possible, convincing. 


4.2 No Reason For Is a Reason Against 
Next, consider the following argument: 


You are interested in the question whether or not God 
exists. To address this question, you consider all argu- 
ments for the existence of God you can find in the 
literature. After a careful examination of them, you 
come to the conclusion that none of them is convinc- 
ing. From this observation, you conclude that you 
have a reason against the existence of God because no 
reason for is a reason against. 


This is the No Reason For Argument (NRF). While this rea- 
soning may have some plausibility, it is not clear whether 
the argument is a good argument. To find out, we fol- 
low Eva and Hartmann (2018b) and provide a Bayesian 
reconstruction of the NRF, which proceeds analogously 
to the NAA and allows us to ask whether learning the 
premises of the argument increases the probability of its 
conclusion. 

We consider the hypothesis H and introduce the 
proposition F, which says “I have not yet found a good 
argument in favor of H.” Furthermore, let Y be a propo- 
sitional variable whose values are the propositions Y;: 
“There are exactly i good arguments in favor of H” (i= 0). 
It is plausible that an agent can be uncertain about the 
value of Y. Many rational agents would surely plead 
ignorance as to whether or not there are any undiscov- 
ered good arguments for the existence of God (and, if so, 
how many). Again, knowledge of the value of Y renders 
F independent of H: if I know that there are five good 
arguments for the existence of God, then the fact that I 
haven’t yet found any one of them should be irrelevant 
to my belief in the existence of God. Furthermore, if I 
learn that there are more good arguments for God’s exis- 
tence than I previously thought, then that should raise 
my degree of belief in the existence of God. Finally, the 
more arguments there are for God’s existence, the more 
likely it is that I find one. These considerations motivate 
the following conditions (see unnumbered list below): 


NRF1: Viz0, yi:= PCY) €[0, 1). 

NRF2: hi; := P(H| Y;) are monotonically increasing in i. 
NRF3: fi; := P(F| Yj) are monotonically decreasing in i. 
NRF4: 


H and F are conditionally independent on Y, that is, P(H| Yj, F) = P(A| Yj). 
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NRF1-NRF4 are structurally near-identical to the con- 
ditions imposed on the NAA. The corresponding Bayes 
net model is also analogous. The only difference is that 
the hj are now monotonically increasing in i. Accord- 
ingly, the proof for the following theorem is analogous 
to the proof of theorem 1. 


Theorem 2. Let P be a probability distribution sat- 
isfying the conditions NRF1-NRF4. Then F discon- 
firms H, that is, P(H|F) < P(H), iff there exists a pair 
(i,j) with i > j such that (1) y; y; > 0, (2) fi> fi, and (3) 
hy < hy. 


We contend that theorem 2 constitutes, under certain 
special circumstances, a full Bayesian vindication of the 
NRF argument. On the basis of this reconstruction, one 
can analyze how good specific NRF arguments are (for 
details, see Eva & Hartmann, 2018b). 


4.3. Toward a General Theory of Reasoning 

and Argumentation 

The examples given in the last two subsections illus- 
trate the slogan “Argumentation is learning” mentioned 
above. Both examples involved the learning of a propo- 
sition, and the learning was modeled using Condition- 
alization. Note, however, that many argument schemes 
involve indicative conditionals of the form “If A, then 
C” as premises, which are notoriously difficult to deal 
with. In the introduction to this section, we represented 
an indicative conditional by the corresponding material 
conditional ~AvC. This was convenient, as it allowed 
us to condition on it. However, the material conditional 
faces many problems, and it is even debated whether 
indicative conditionals can be modeled as propositions 
at all (for details, see Douven, 2018, and chapter 6.1 by 
Starr, this handbook). It is therefore advisable to look 
for a more general updating method that can be applied 
to both propositional and nonpropositional evidence. A 
general theory of Bayesian argumentation cannot work 
without such a method. To address this problem, Eva 
et al. (2020) develop the distance-based approach to Bayes- 
ianism that builds on the abovementioned principle of 
Conservativity. Learning the indicative conditional “If 
A, then C” from a perfectly reliable information source 
then suggests the constraint P’(C| A) = 1. The full poste- 
rior probability distribution P’ follows by minimizing a 
suitable distance measure (such as the Kullback-—Leibler 
divergence) between P’ and P. Eva and Hartmann (2018a) 
apply this proposal to Bayesian argumentation and show 
how Bayes nets can be used to model, for example, dis- 
ablers in an argument. 
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5. Outlook 


In this chapter, we have introduced the theory of Bayes 
nets, shown how Bayes net models can be constructed 
within the Bayesian framework, and presented a num- 
ber of examples from the philosophy and psychology of 
reasoning and argumentation that illustrate the power 
of the approach. However, Bayes nets have also been 
used in other areas of rationality research (such as the 
study of decision making), and there is no doubt that a 
whole range of further problems can be addressed with 
this methodology. 

It is also worth noting that Bayes nets can be used with 
other theories of uncertainty as well. For example, Spohn 
(2012, chapter 7) shows that ranking theory satisfies the 
semi-graphoid axioms. Other authors investigate condi- 
tional independence structures for imprecise probabili- 
ties (see, e.g., Cozman, 2012; Halpern, 2003, chapter 4). 
Hence, nothing hinges on the Bayesian framework that 
we used in this chapter. At the same time, it is true that 
most applications of Bayes nets in rationality research 
studied so far presuppose the Bayesian framework, which 
is easy to use and has a solid normative foundation. It 
would be interesting to construct analogous models in 
other frameworks and to compare the resulting analy- 
ses with the corresponding Bayesian analyses (see also 
Colombo, Elkin, & Hartmann, in press). This will lead to 
further progress in rationality research. 
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Notes 


1. For a textbook exposition, see Darwiche (2014, section 4.4) 
and Pearl (1988, section 3.1). 


2. We follow the convention, adopted, for example, in Bovens 
and Hartmann (2003), to represent propositional variables in 
italics and their values in roman script. For instance, the vari- 
able A has the values A and 7A. Here and in the remainder, we 
also use the shorthand notation P(A, B) for P(A» B). 


3. If Aj is a root node, then pa(A,) is the empty set (@) and 
P(A; | pa(Aj)) = P(A). 


4. We sometimes use causal language when talking about 
various Bayes nets. And indeed, causal intuitions help when it 
comes to construct a Bayes net model, which typically respects 
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the “causal direction.” For instance, we always draw an arrow 
from the hypothesis variable to the corresponding evidence 
variable. Note, however, that all we need for the applications 
discussed in this chapter is that a Bayes net represents a joint 
probability distribution. Hence, the causal language is, strictly 
speaking, only of heuristic value here (see chapter 7.1 by Pearl, 
this handbook). 


5. Section 5 of this volume surveys the various contenders. 
See also chapter 4.7 by Dubois and Prade (this handbook) and 
Halpern (2003). Oaksford and Chater (2007) discuss topics in 
the psychology of reasoning from a Bayesian point of view. 
Hajek and Hartmann (2010) and Hartmann and Sprenger 
(2010) survey Bayesian epistemology, and Sprenger and Hart- 
mann (2019) investigate various topics in the philosophy of 
science from a Bayesian point of view. 


6. “Relevant” means relevant for the specific problem or ques- 
tion the agent is interested in. 


7. It would be very helpful to have an automated way to extract 
the relevant variables and their relations from potentially large 
data sets. For attempts in this direction, see Chalupka, Bischoff, 
Perona, and Eberhardt (2016). 


8. The choice of the variables, and which relations between 
them one assumes, may also be suggested by considerations 
about which Bayes net provides a good representation. 


9. See Frigg and Hartmann (2016/2020) for a survey of the cor- 
responding philosophy of science literature. 


10. For details and an explanation, see Bovens and Hartmann 
(2003). 
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4.3 Evidential Relevance 


Arthur Merin 


Summary 


Signed evidential relevance among truth-valuable prop- 
ositions has qualitative, comparative, and quantitative 
aspects. We explore the abstract theory of qualitative 
relevance quite fully, notably so the role of conditional 
irrelevance. Regarding ordinal and quantitative aspects of 
relevance, we offer a notational proposal and two exten- 
sions of the state of the art. First, we demonstrate task- 
related conditions for ordinal equivalence of all relevance 
measures satisfying a highly intuitive desideratum shared 
by the major relevance measures. This affords a useful 
degree of measure independence for many comparative 
relevance judgments. Second, we elicit relevance proper- 
ties of disjunctive evidence to complement familiar facts 
about conjunctive evidence. The central role of relevance 
for reasoning is emphasized throughout. 


1. Intuitions of Evidence in Argumentation 


Will some proposition or statement, A, speak for the truth 
of another proposition, B, or against it? Questions of this 
kind are ubiquitous, and people answer them within the 
context of their general beliefs about the world. Example: 
Let A = “There are dark clouds in the sky today,” B, = “It 
will rain today,” Bz = “It will be dry today,” B3 = “The 
Dow Jones will rise today.” In typical belief contexts, A 
will speak for B,, against Bz, and neither for nor against 
B3. Similar relations obtain between statements about past 
events, as well as between possible experimental results 
and timeless scientific hypotheses. 

Here is a way to reconstruct this kind of reasoning. 
We are uncertain about the truth values, True or False, 
of By, Bz, and B3. We may be uncertain about the truth 
value of A. And we can distinguish, in principle, at least 
a handful of degrees of credence between certainty of 
truth and certainty of falsity. Now comes the principal 
ingredient of the reconstruction. If, being uncertain 
about A (“dark clouds”), we learnt for certain that A is 
true—just this much!—then our degrees of credence 
should move upward for B, (“rain”), go downward for Bz 


(“dry”), and remain unchanged for B; (“Dow up”). We 
need not actually become certain that B; is true. Neither 
need we become certain that B, is false. These predica- 
ments are but special cases of “speaking for” and “speak- 
ing against.” 

Next suppose that our beliefs are as before, but that 
we learn that A (“dark clouds”) is false; equivalently, -A 
(“no dark clouds”) is true. The suggestion here is as fol- 
lows. Our credence in B; (“Dow up”) would again remain 
unaffected. Our degree of credence for Bz (“no rain”) 
would go up. And our credence in B; (“rain”) would go 
down. If so, learning that A is false must afford a basis for 
determining what A speaks for, against, and neither way. 
A will speak for B if A speaks against B. 

In the diction of jurisprudence and of science, “speak- 
ing for” B means being evidence for B. “Speaking against” 
B means being evidence against B. In this evidential sense, 
we say that A is positively relevant (“positive”) to B in a 
credal context or state Cred if and only if (iff) it speaks 
for B in Cred, negatively relevant (“negative”) to B iff it 
speaks against B in Cred, evidentially irrelevant to B if 
it speaks neither for nor against B in Cred, and relevant 
iff not irrelevant. 

Plausible as the idea may appear, it still remains vague 
and prone to apparent exceptions. Some framework for 
specifying it is needed to give it substance. 


2. Evidential Relevance Explicated 
in Probability Theory 


To turn intuitions about relevance into a theory, one inter- 
prets degrees of credence in A, B, and so on in a credal 
state, as credal probability values P(A), P(B), and so forth." 

Changes of credal state, from Cred to Cred’, are then 
represented by changes of probability measures, from P 
to P’. Any such change will yield credal reassessments 
P'(A), P'(B), and so on. However, it is one particular belief 
change regime that yields a workable explication of the 
intuitive notion of relevance. This regime is intimately 
associated with the concept of conditional probability, 
in terms of which relevance is standardly definable. 
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Definition 1. The conditional probability of A given B, 
notated P(B| A), is defined by P(B|A) =a: P(AB)/P(A) 
whenever P(A) 0, else undefined.” 


We gloss it, roughly for now, as the probability that B 
would have if A were assumed. Recall now section 1. Sup- 
pose we learn A, no more, no less, and without any doubts 
or reservations. Then we should be in a new, “posterior” 
credal state, P’, validating P’(A) = 1; P'(-A) = 0. Let our 
update be by conditioning P on A. Then the posterior state 
P' may be written P,, and for all B, the posterior probabil- 
ity of B equals its conditional probability: P,(B) = P(B| A). 

The probability-change explication of speaking “for” 
or “against” makes A positive to B in P iff P4(B) > P(B) 
and negative to B iff P4(B) < P(B). This yields 


positive 
irrelevant +} to B (in P) iff 
negative 


Definition 2. A is 


P(B|A) > P(B) 
P(B|A) = P(B) or P(A) = 0 }. 
P(B|A) < P(B) 


Multiplying the (in)equalities of definition 2 with P(A) 
yields the equivalent definition 3, which we state with 
obvious abbreviations and using the standard symbol 
(A 1 B)p for “A is irrelevant to B in P”: 


pos 
Definition 3. Ay + 
neg 


B iff P(AB) = P(A): P(B). 


Definition 3 depends on definition 2 for the underlying 
intuition. In return, it needs no completion clause and 
reveals the symmetry of relevance relations. 


3. Properties of Qualitative Relevance 
and Extreme Positivity 


Qualitative relevance of A to B, and of B to A, is specified 
only in terms of polarity. I present six of its properties in 
order of intuitive simplicity or importance. 

(1) Symmetry of qualitative relevance: Explicated in prob- 
ability theory, positive relevance, negative relevance, and 
irrelevance are symmetric relations: 


pos pos 
Theorem 1. A} L +B iff By L A. 
neg neg 
P P 


The proof is obvious from definition 3. For instance, 
if A speaks for B, then B speaks for A, however weakly 
(intuitively). 
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(2) Polarity reversal by negation: Negating just one of 
A and B reverses relevance polarity and preserves irrel- 
evance. Negating both of A and B leaves relevance polar- 
ity unchanged: 


pos neg neg 
Theorem2. A, L > B iff -Ay 1 > B iff Ay L 
neg pos pos 
P P P 
pos 
“=B iff -Ay Lt ¢-B. 
neg 
P 


(3) Certainty implies irrelevance: Uncertainty is a neces- 
sary condition for relevance: 


Theorem 3. A proposition A that has extreme probabil- 
ity, P(A) = 0 or =1, is irrelevant, under P, to any other 
proposition and to itself. 


For proof instantiate definition 3. To be relevant (or for 
anything to be relevant to it) in P, a proposition A must 
therefore be P-contingent, 0 < P(A) < 1. 

(4) Extreme positive and negative relevance: Classical 
logical entailment, AFB, is transitive: AF B and BEC 
together imply A FC. Positive relevance is not in general 
transitive. That is, A posp B and B posp C need not imply 
A posp C. An example will be given shortly. However, 
there are conditions under which positive relevance is 
transitive. One special case thereof engages “conclusive- 
evidence-for” as explicated in definition 4a: 


Definition 4 (Carnap 1950, modified). (a) A is extremely 
positive to B iff P(B) < P(B| A) = 1. (b) A is extremely neg- 
ative to B iff extremely positive to ~B. We abbreviate 
(a) as A exposp B and (b) as A exnegp B. 


Clause (a) succinctly describes in a credal probability 
theory what is conveyed when someone asserts, without 
reservations, an indicative conditional “If A, then B.” Of 
course, such an assertation can also be made with less 
certainty. Extreme positivity explicates what colloquial 
language mostly means by “entailment.” The idea is that 
A is a compelling and, in this sense, sufficient reason for B. 
It is sufficient by P(B| A) = 1, which makes B a conditional 
certainty. It is a reason in virtue of P(B) < P(B|A). Reason- 
hood has a reflex in our sense of evidential aesthetics: 
“Tf the Dow rises (=A), (then) 7 is a prime number (=B)” 
sounds nonsensical even when P(A) >0 is uncontrover- 
sial, as normally it is, and P(B| A) = 1 must therefore hold 
because P(B) = 1 holds. Classical entailment, At B, is 
related to extreme relevance by 


Lemma 1. If 0 < P(A), P(B) < 1, and A entails B (AFB), 
then A exposp B. 
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Entailment, |, is a transitive relation. So is its proba- 
bilistic counterpart, P-entailment, defined by Af pB =ar 
P(A —B) = 1 (where — is material implication). Extreme 
positivity inherits the transitivity of F and Fp: 


Theorem 4. Extreme positive relevance is transitive: if 
A exposp B and B expos; C, then A exposp C. 


(5) Conditional irrelevance and conditional transitivity 
of positive relevance: When positive relevance of A to B 
and of B to C is short of extreme, transitivity may fail. 
Here is an example: Assume A exposp B, C exposp B, and 
P(AC) = 0. These assumptions are easily satisfiable. For 
example, A = “Trump has won,” B = “The U.S. has a new 
president,” and C = “Clinton has won.” Then A posp B, 
B posp C (because of symmetry), yet A exnegp C, hence 
A negp C. 

Transitivity is, however, ensured by a supremely use- 
ful property. Relevance relations can be conditional on 
a proposition. On noting that P(A) = P(A|T) is a prob- 
ability theorem (where T is any tautology), definition 3 
becomes a special case of 


Definition 5S. A is irrelevant to (independent of) C condi- 
tionally on Bunder P, (A LC | B)p, iff PAC | B) = P(A|B)- 
P(C |B). When each of (A 1 C|B)p and (A LC| 7B)p 
holds, we write (A 1 C | + B)p. 


We now have a sufficient condition for transitivity of 
positive relevance that need not be extreme: 


Theorem 5. If A and B are irrelevant to one another 
conditionally on B and on 7B, that is, if (A LC | +B), 
then mutual positivity of A and B and mutual positiv- 
ity of B and C imply mutual positivity of A and C.* 


Condition (ALC|+B)p becomes more intuitable on 
realizing that the extreme-relevance configuration of 
A exposp B and B exposp C entails the antecedent of the 
following lemma and thus (A 1 C | + B)p: 


Lemma 2. If P(B|A) = 1, P(C|B) = 1, and P(B) < 1, then 
(A LC|+B)p. 


Hence, theorem 5 properly generalizes intuition-friendly 
theorem 4. 

(6) Conditional irrelevance and paradoxa of relevance: 
Theorem 5 shows that (A 1 C | + B)p does not imply (A L C)p. 
This counterintuitive fact constitutes an instance of 
a well-known statistics paradox. Suppose B and —B spec- 
ify complementary properties of members of a select pop- 
ulation, say, “British” and “non-British.” Let A stand for 
having been administered pharmaceutical ©, and C for 
being cured of disease A. These properties and their com- 
plexes can be turned into propositions (A, B, B,C, etc.), 
whose credal probabilities reflect statistical frequencies. 
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(Think of proposition D as “t has property D,” where « is 
a randomly sampled individual.) Now suppose we inves- 
tigate the relevance of A to C. Then theorem 5 implies 
that A may be positively relevant to C in the popula- 
tion as a whole, but irrelevant to C in each of subpopu- 
lations B and —B (because, e.g., the drug ® is actually 
inefficacious concerning A, but both the drug adminis- 
tration and the spontaneous remission rate are higher 
among British than among non-British citizens). This is 
an instance of “Simpson’s Paradox” (see chapter 7.1 by 
Pearl, this handbook). 

In return, issue-conditional independence (A 1 C | + B)p 
(referring to the issue whether B or ~B) precludes other- 
wise possible, intuitively paradoxical relevance configu- 
rations such as the one where A and C are positive to B, 
but AC is irrelevant or negative to B (cf. Carnap, 1950, 
chapter 6). For Reichenbach (1956) shows, as a special 
case of a result for various combinations of relevance 
polarities: 


Theorem 6. Suppose that (A 1 C|+B)p holds, A and B 
are mutually positive, and so are B and C. Then B and 
the conjunction AC are mutually positive. 


Reichenbach’s intended interpretations for theorems 
5 and 6 had B as a common cause of A and C. Pearl (1988; 
see chapter 7.1 by Pearl, this handbook) surmised that 
medical diagnostic categories B (diseases, syndromes) are 
formed to make symptoms A and C conditionally inde- 
pendent given + B, by identifying B as their common 
cause.” A test of Pearl’s thesis would thus be to search for 
diseases that exhibit Carnap-paradoxical symptomatolo- 
gies. Anyone’s credence in the thesis should increase if 
none were found. But why? Simply speaking, because 
finding a disease with a paradoxical symptomatology 
would be extremely negative to the thesis, which implies 
that finding none is positive to it. We now address this 
idea more generally. 


4. An Application of Qualitative Relevance Reasoning 


The most familiar applications of theorem 1 (symme- 
try) and theorem 2 (polarity reversal under negation) 
presuppose a sorting of propositions into two types: (1) 
“hypothesis” statements or “theories,” labeled “H,” and 
(2) observational “data reports” or “evidence statements,” 
labeled “D” or “E.” Suppose a hypothesis H having non- 
zero probability entails some data report D, which might 
be one possible outcome of an intended, future investiga- 
tion. In symbols: Ht D. Then P(D|H) = 1 holds. Hence, 
provided 0 < P(D), P(A) <1 holds, H is extremely positive, 
and thus positive, to D. By symmetry, P(H | D) > P(H) must 
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also hold. Suppose outcome D is then observed and P 
conditioned on it. Then Pp(H) > P(H) must hold. H has 
become more credible. Alternatively, suppose outcome ~D is 
observed. By theorem 2, P.p(H) < P(A) holds, so H becomes 
less credible. This is what the relevance laws developed so 
far entail. 

Complications arise when (a) D is already-known, 
“old” evidence to be explained, rather than predicted, 
and (b) P is required to reflect this knowledge. By theo- 
rem 3, such D must be P-irrelevant: Pp(H) = P(A) (Glym- 
our, 1980). A standard example has H instantiated by 
Einstein’s theory of general relativity, which entailed D, 
a long-known and seemingly anomalous fact about the 
orbit of planet Mercury. This relationship was widely 
judged to increase confidence in H. Reconstructions of 
this judgment appeal to hypothetical, partly noncondi- 
tioning update histories (Jeffrey, 2004), introduce prob- 
abilities over primitive propositions about entailment or 
support (cf. Earman, 1992; Fitelson & Hartmann, 2015), 
or leave D slightly in doubt and choose a relevance mea- 
sure (see below) that gives near-certain D an appreciable 
amount of relevance (cf. Hajek & Joyce, 2008). Working 
statisticians have, in this respect, an easier life. Faced 
with known data, they usually draw on ready-made 
probability models for conditional statistical frequen- 
cies of occurrence. This replaces commitment (b) by 
appropriate probability distributions for hypothetical 
predictions under repeatable conditions.* 


5. Comparative Strength and Amount of Relevance 


Consider propositions A; = “There are dark clouds,” 
Az = “The weather forecast predicts rain,” A; = “The 
weather forecast predicts dry conditions,” and B = “It will 
rain.” Imagine, as seems natural, that each of A; and A 
would speak for B, while A3 speaks against B. Now sup- 
pose that you wonder which of A; and A2, if any, would 
speak more strongly for B than the other does. Or imagine 
that you learn A; and A; and wonder if A,A; speaks for 
B, or against B, or does neither. Or imagine asking which 
of B (“It will rain”) and B’ (“It will be snowing”) is more 
strongly spoken for by A, (“dark clouds”) in a given credal 
state. Neither of these questions can usually be resolved 
by purely qualitative considerations. 

Suppose, then, that A; and A, are each positive to B 
in credal state P. If we compare A; and A; by strength 
of relevance to B, the resultant ordering within the 
probabilistic explication will, in general, depend on 
measurements of strength. This fact already suffices 
to make measures of relevance a pertinent topic for 
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anyone interested in probabilistic relevance. Rankings 
may indeed depend on our choice of measuring func- 
tion. This is much as could happen in ranking compa- 
nies by asset value: rankings may differ according to the 
accounting convention used. 

A further problem arises. If A and B are mutually rel- 
evant, A may be more strongly relevant to B than B is 
to A under one measure, less relevant under another, 
and equally relevant under a third. A helpful measure- 
ment notation should thus make the intended direction 
immediately clear, without tacitly relying on the classifi- 
cation of content as, say, evidence or hypothesis, which 
is often application dependent (see below). What moti- 
vates conventional assignments of relevance direction 
may then be freely investigated. 


5.1 Notation and Terminology 

In recent literature, degree of relevance is widely glossed 
as “amount of ‘(incremental) confirmation’ of ‘hypothesis’ H 
provided by ‘evidence’ E under probability measure P and 
measure of type C.” Notation is often “C)(H,E),” some- 
times “C)(E,H).” The Cp(-,-) format affords no natu- 
ral mnemonic of direction. True, its gloss will suggest 
a direction: from “evidence” to “hypothesis.” Yet often 
enough we reason from hypotheses to possible data or 
find no warrant for the evidence/hypothesis type distinc- 
tion. Thus, reliance on E and H abets premature identi- 
fication of structural argument position with content type. 
Concomitantly, labels such as “confirmation,” “corrobo- 
ration,” and “support” need a prefix “incremental” to 
avoid being misunderstood as designating conditional 
probability, P(H| E). They also sound awkward with the 
prefix “negative.” “Relevance” (Keynes, 1921; Carnap, 
1950; Jeffrey, 2004) has no such problems. 

All this speaks for a content-neutral approach to sym- 
bolization, at least alongside its common philosophy-of- 
science instantiation. I propose an iconic, signpost-like 
separator symbol for proposition arguments that indi- 
cates conventional measurement direction as the word- 
ing does in “relevance of A to B.” 


Definition 6 (Notation, Terminology). Let Ry(A >B)p 
designate—subject to conditions specified in defini- 
tion 8—the numerical amount of relevance of propo- 
sition A to proposition B as assessed in terms of a 
relevance measure of type T under a credal probabil- 
ity measure P. Refer to the proposition arguments A 
and B preceding and following the signpost separator 
> as being respectively in source position and in target 
position. 
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Source > target direction affects numerical measure- 
ment values, at some point or other, for all relevance 
measures that do not have the property introduced in 


Definition 7. A relevance measure Ry(A > B)p is amount- 
symmetric iff Rt(B > A)p = Ry(A > B)p for all A,B, P. 


Amount-symmetric Rr are, in this unambiguous sense, 
nondirectional. 


5.2. Two Definitional Conditions on All 

Relevance Measures 

Many distinct measures of relevance have been discussed 
in the literature (see, e.g., Eells & Fitelson, 2002; Crupi & 
Tentori, 2016). All measures validate 


Definition 8. A function R;(A > B)p mapping pairs 
(P, (A, B)) consisting of a probability measure P and 
a pair (A,B) of propositions in the domain of P to 
numeric values is a probabilistic relevance measure iff 
it satisfies 


(a) Transparency: R1(A > B)p must be a smooth (i.e., 
differentiable) function of the probability dis- 
tribution P over the atoms AB, An 7B, ~AaB, 
-Axn 7B of the propositional algebra generated 
by A and B, and its value range must be a subin- 
terval, proper or improper, of the extended real 
numbers R* = RU {-c, oo}.° 


D: R,(E>HA)> =5 


P(H|E)__ P(EH) 
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(b) Quality Representation: Ry(A > B)p must repre- 
sent qualitative relevance polarity: its value range 
must contain an element u := 4 such that 


pos 
Re(A>B)p = wy iff Ay L 7B. 
neg 
P 

Measures in actual use have for neutral value either 
u= 0, the “neutral” identity element for addition, ors = 1, 
the multiplicative identity. Logarithmic transformations 
map multiplication to addition and thus multiplicative 
to additive identities. 

In what follows, we mostly adopt the common for- 
mat R(E > H), which identifies the “source” proposition 
with “evidence” E and the “target” proposition with a 
“hypothesis” H. This makes for easy cross-reference to 
the confirmation literature, just as A and B do to the 
content-neutral mathematical literature. 


5.3 The Big Three Measures of First Resort 

and Their Ordinal Equivalents 

The “Big Three”—D, Q, and L—are the oldest and most 
commonly used relevance measures. Their formal defi- 
nitions are given below. (D, Q, and L stand for “Differ- 
ence,” “Quotient,” and “Likelihood.”) Also given are the 
most useful and most common transforms for Q and L, 
namely, QL and LL. 


P(H|E)- P(H), if P(E) #0, 
O, else. 


PEI) se pce), P(H) #0, 


Q: Re(E>~H)p =a 


P(E|H) 


P(E|AH)’ 
L: R\(E>A)p =o 


1, else. 


P(H) ~ P(E)P(H) 


P(E) ’ 
1, else. 


if P(H), P(aH), P(E| nH) #0, 


co, if P(H), P(aH)#0 and P(E| nH) =0#P(E|H), 


QL: Ra(E > H)p =a log Ro(E > H)p = log P(EH) - log P(E) - log P(A). 


LL: R,(E>H)> 


log R, (E>), = log P(E|H) — log P(E|=H), 


if P(H), P(AH), P(E|H), P(E| nH) #0, 
co, if P(E| AH) =0# P(E|H), 
—oo, if P(E|H)=0#4P(E|4H), 

O, else. 
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Let me introduce two further measures to be men- 
tioned below: KO and Z. Formal definitions for these are 
given at the bottom of this page. 

Even more measures are discussed in the literature. 

The “difference measure” Rp (Hugh MacColl’s 6 of 
1897 or earlier; cf. Keynes, 1921, pp. 154-155) has irrel- 
evance value 1 = 0 and range (-1,1). The “coefficient of 
influence” (W. E. Johnson, 1921 or earlier, published in 
Keynes, 1921) or “relevance quotient” Q’ has 1 = 1 and 

P(EH) 
P(E)P(H) 
amount symmetry: Ro(E > H)p = Ro(H > E)p. The “likeli- 
hood ratio” L has « = 1 and range [0,]. Modulo loga- 
rithms, L was commended by C. S. Peirce, A. M. Turing, 
and I. J. Good. It has an information-theoretic claim to 
primacy, being both (i) sufficient and (ii) necessary to 
capture all the information in the data E that bears on 
the hypothesis bipartition {H, =H}. It conveys all of it 
(i) and any other sufficient statistic factors through it 
(ii) (cf. Savage, 1954, pp. 128-140). 

The logarithmic QL-transform (Good, 1950) has 1 = 0 
and range [-~,«). The “log-likelihood ratio” LL has 
t= 0 and range [--,] and is widely used in statistics. 
The “sigmoid” transform KO (Kemeny & Oppenheim, 
1952) of LL has « = O and squashes the range to [-1, 1]. 
Finally, for the measure Z (Crupi & Tentori, 2013) the 
irrelevance value zis O and the range is [-1, 1]. The vari- 


range [0, ~). Its variant expression reveals its 


ous transforms’ existence suffices to motivate a highly 
general concept. 


Definition 9. Measure types T and T’ are ordinally equiv- 
alent (T~, T’) iff, for all possible P and propositions 
A,B,C, D in the domain of P, Rr(A > B)p z Ry(C > D)p iff 
Rr(A>B)p = Rr(C>D)p. Equivalently, T~, T’ iff there 
exists some strictly monotonically increasing (isotonic) 
function f for which T' = f (T).” 


Distinct, but ordinally equivalent, measures may nonethe- 
less differ in convenience of range and, more important 


KO: Ryo(E>H)>p 
P(E|H)- P(E|=H) _ 


P(EH) — P(E)P(H) 
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for cognitive-engineering applications, in various alge- 
braic properties. 

The Big Three have appealing interpretations as addi- 
tive or multiplicative operators. They map the prior cre- 
dal value of a “target” proposition H to its value in the 
posterior credal state obtained by conditioning the prior 
P on the “source” proposition E. For D and Q, the credal 
values are probabilities. For L they are odds, O(H) and 
O(H | E), from which probabilities may be recomputed.'® 


P(H| E) = P(H) + Ro(E > Hp, (1) 
P(H| E) = P(H) “RoE > Hr, (2) 
log P(H| E) = log P(H) + Ra(E > H)», (21) 
O(H| E) = O(H) RUE > H)p, (3) 


log O(H| E) = log O(H) + Ru(E > H)p. (3L) 


Observe that equations (2) and (3) express, respectively, 


P(E|H)P(H) 


Bayes’ Theorem: P(H|E) = P(E) ; 


Bayes’ Theorem in odds form: 
P(H|E) _ P(H) P(E|H) 
P(AH|E) P(aH) P(E|nH) 


5.4 Axiomatic Characterization 
A most illuminating characterization of the Big Three— 
although only up to ordinal equivalence—is given by 
Crupi, Chater, and Tentori (2013) and Crupi and Tentori 
(2016). Modulo labeling, they consider four axioms for 
measure types T: 
Posterior Probability Decisiveness (POP-D): For all H, E1, E2, 
P: Ry(E, > H)p = Ry(E2 > H)p iff P(H|E,) = P(A|E2). 
Likelihood Decisiveness (LIK-D): For all E, Hi, H2, P: 
Ry(E > Hi)p = Rr(E > Ho)p iff P(E| Hi) = P(E|H2). 
Modularity for Conditionally Independent Evidence (MCIE): 
For all A, Ey, Eo, P: Rr(E2 > H)p, = Rr(E2 > H)p whenever 
(E; L Ez |+H)p, where, as before, Pz =a: P(-|E), that is, P 
conditioned on E. 


if P(E), P(E), P(A), P(AH) #0, 


at 


P(H|E) - P(H) 
1-P(H) 
P(H|E) — P(H) 
P(A) 


Z: Rz(E > H)p =a 


O, else. 


P(E|H)+P(E|=H)  P(H)P(EsH) + P(aH)P(EH)’ 


O, else.” 


, if P(E) 40, P(H) #1, and P(H|E) > P(H), 


, if P(E), P(H) #0 and P(H|E)< P(H), 
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Modularity for Disjoint Hypothesis Disjuncts (MDHD): For 
all E, Hh, Hp, P: if P(H\H2) = 0, then Ry(E > Hi v H2)p = 
Ry(E > Hy)» iff P(H2 | E) = P(A2)."' 


Then they derive a representation theorem: 


Theorem 7 (Crupi et al., 2013; Crupi & Tentori, 2016). 
Let Rr be a real-valued function satisfying Transpar- 
ency (definition 8a) with the possible exception of its 
smoothness requirement. Then Rr is identical to, or is 
an isotonic transformation of, 


(i) Rp iff it validates POP-D and MDHD, 
(ii) Rg iff it validates POP-D and LIK-D, 
(iii) R, iff it validates POP-D and MCIE. 


The challenging “if”’-parts of theorem 7 yield ordinal 
equivalence classes [T] =a: {T’| T’ ~; T} of measures T.'” To 
determine specific measures D, Q, L, QL, LL, KO, and Z, fur- 
ther conditions are needed, among them “smoothness.” 

We now discuss the four axioms. Note first and impor- 
tantly that POP-D and MCIE legislate for measurements 
comparing relevance of two “source/evidence” proposi- 
tions E,, E, to a given “target/hypothesis” H, while LIk-D 
and MDHD apply to comparisons of hypotheses H,, H2 
for given E. Thus, POP-D and LIK-D do not compete and 
are indeed each validated by Q. 

POP-D seems mandatory to most commentators. Its 
denial feels counterintuitive, and POP-D has also been 
argued to be necessary for enumerative induction based 
on the proportion of favorable and unfavorable out- 
comes of a series of structurally identical experiments 
(Steel, 2003). Only measures validating POP-D should 
thus be attractive as measures of first resort.’ 

LIK-D is often dubbed the “Law of Likelihood.”"* LIK-D 
feels less compelling than POP-D. However, intuitions on 
single axioms have limited normative value. Users who 
like POP-D and find direction-assignments unwarranted 
will adopt LIK-D, because POP-D and amount symmetry 
entail LIK-D. Users who value POP-D and prefer nonsym- 
metry at least for extreme relevance must allow violations 
of LIK-D. 

MCIE, found in Good (1960) and Heckerman (1988), 
has its much desired special case in the respective con- 
ditional multiplicativity of L and additivity of LL. For, 
we have 


Lemma 3: If (Ey tL E> | +H) then RUE: E2 > A)p = 
RE, > H)p> Ri(E2 > H)p and thus Ru(E:E2 > H)p = 
R(Ei > H)p + Ru(E2 > H)p. 


These practical benefits of MCIE are unavailable for other 
isotonic transforms of L (e.g., KO). 
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MDHD looks sensible but faces an embarrassment (cf. 
Crupi & Tentori, 2016). Suppose E is extremely negative 
to H. Now weaken H to H' = Hv G, by a disjunct G that is 
P-irrelevant to E and P-disjoint from H, that is, P(GH) = 0. 
Then Rp(E > H)p = Ro(E > H')p, even though E speaks con- 
clusively against H but only inconclusively against H’. 
Since the L- and Q-measures do not face this problem, 
the property illustrates ordinal inequivalence of D to 
each of them. 


5.5 Measure Dependence and Taskwise Independence 
To what extent are purely ordinal assessments of com- 
parative relevance measure dependent? The Big Three, 
D, Q, L, are known to be pairwise ordinally inequiva- 
lent. To localize the inequivalences, I offer the following 
results. 


Theorem 8. For the task of ranking multiple “target” 
propositions H; (j =1,...,m= 2) by strength R(E> Hj)p 
of relevance of a given “source” proposition E to each 
of them, measures D, Q, and L are pairwise ordinally 
inequivalent. 


Theorem 9. For the task of ranking multiple “source” 
propositions FE; (i=1,...,n2=2) by strength R(E; > H)p 
of relevance to a given “target” proposition H, all rel- 
evance measures validating POP-D (e.g., D, Q, L, QL, 
LL, Z) are pairwise ordinally equivalent. 


Theorem 9 holds because POP-D prescribes an ordinal 
assessment schedule for any fixed target H and variable 
pair of source propositions Ej, £;. Indeed, theorem 9 and 
the nondirectionality of amount-symmetric Q (section 5.3) 
entail 


Lemma 4. If POP-D is a soundness requirement on 
measures R;7(E > H)p, then sound ordinal ranking of 
“source” propositions E£; (i= 1,...,n2=2) by strength 
of relevance to a given “target” proposition H does 
not depend on the direction of measurement. 


This lends credibility to the probabilistic explication of 
many judgments of comparative relevance that might 
otherwise seem critically measure dependent. Applications 
include solutions to Hempel’s “Raven Paradox” (Hosiasson- 
Lindenbaum, 1940; more accessibly in Earman, 1992; 
Hajek & Joyce, 2008; Crupi, 2015). In abstract outline, this 
is the following predicament: 


(1) A (“All ravens are black”) is supported by the con- 
junctive proposition A = A,A, (where A; = “a is a 
raven” and A: = “a is black”). Here “a” stands for a 
randomly encountered physical entity. 
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(2) A logical equivalent, G (“All nonblack things are 
nonravens”), of H is supported by conjunctive prop- 
osition B= B,B, (where B, = “a is not black” and 
B, = “ais not a raven”). 

(3) G is entailed and thereby supported by C, “a is a 
white shoe.”!® 


The problem arises because a plausible deductivist cri- 
terion of evidential support prescribes treating logical 
equivalents, here H and G, alike for support by evidence. 
Hence B (and, with mild further assumptions, C), in sup- 
porting G, should support H. But this is counterintuitive. 
Probabilistic proposals observe that B,, “ais nonblack,” is 
vastly more probable than Aj, “a is a raven,” since there 
are vastly more nonblack things than ravens. Concomi- 
tantly, B will be vastly more probable than A. This perti- 
nent observation and the resulting unsurprisingness of 
B make for its low relevance. To resolve the paradox for- 
mally, the known resolution proposals must also make 
some irrelevance assumptions. Hosiasson-Lindenbaum 
(1940, p. 138, ll. 5-8) assumes H to be probabilistically 
independent from each of A; and B;. All known propos- 
als use some familiar relevance measure or other to con- 
clude that A and B are each positive to H, although B is 
positive only to a negligible degree. This explains the 
pretheoretical intuition that B is irrelevant to H, which 
extends to the irrelevance of C in the augmented story. 
Theorem 9 assures us that the suggestive ordinal result 
“B is less positive to G or H than A” holds under any pos- 
sible measure satisfying POP-D."° 


5.6 Features-in-Use of Various Relevance Measures 
Measures contrast by various features that may be 
found desirable or undesirable, absolutely or in given 
uses, and normatively or descriptively. For instance, D 
must equally value prior-to-posterior shifts 0.2+~0.4, 
0.5+>0.7, and 0.8+>1. By contrast, L- or LL-values for 
a target probability that increases from 0.999 to cer- 
tainty, 1, must be (infinitely) higher than for an increase 
from 0.001 to 0.999. In this subsection, some properties 
affecting purely ordinal comparisons are discussed: 


Extreme valuation of extreme relevance: Measures L, Z, and 
their ordinal equivalents attain the maximal (respec- 
tively, minimal) values of their ranges if and only if 
relevance is extremely positive (respectively, extremely 
negative). This property pleases anyone to whom con- 
clusive argument is the strongest argument. 

Amount symmetry: Measures Q and QL are amount- 
symmetric (section 5.1). So is the measure Rc(E > H)p =ar 
P(EH) - P(E): P(A) (Carnap, 1950), which, however, 
violates POP-D. Among measures that are not 
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amount-symmetric, D always makes the less probable 
of mutually relevant A and B more relevant by abso- 
lute amount to the more probable than vice versa. For- 
mally: If not (ALB)p, then|Rp(A > B)p| = |Ro(B > A)p| 
iff P(A) = PB). L works likewise, except for extreme 
negativity.” 


Intuitions on amount symmetry are strong only in 
the case of asymmetric extreme positivity. Example: Let 
A = “15 divides k,” and B = “3 divides k,” where k is only 
known to be a natural number = 15. Evidently enough, 
A entails B, B does not entail A, and indeed P(A) < P(B). 
We now have, for instance, Rp(A > B)p > Rp(B > A)p, in 
line with A’s unilaterally extreme positivity to B. We also 
have Ro(A > B)p = Ro(B > A)p, which many will balk at, 
because it ignores the entailment asymmetry. 

The next three features concern conditional operation 
homomorphisms, which are sometimes unconditional. 
Homomorphisms are structure-preserving mappings and 
are essential to predictability of many ordinal and all 
quantitative relevance properties of logical complexes. 
Here they map Boolean operations on propositions to 
equally simple, or at any rate to relatively simple, arith- 
metical operations on relevance values: inversion, addi- 
tion, and weighted mean. Any relevance measure will 
afford some homomorphisms conditional on suitably 
narrow constraints, but most such constraints are not 
practically useful. Nor will all homomorphisms, condi- 
tional or otherwise, be desirable in practice. Let us look 
at three of them. 


Relevance-functionality of negation: This refers to the inver- 
sion of relevance values r under negation: to -r when 
tr =0, to 1/r when ty = 1. It engages two distinct fea- 
tures: negation in target position and negation in 
source position. For source proposition E and target 
proposition H, target inversion means evidence E 
always speaks as strongly for hypothesis H as it speaks 
against ~H. Professed opinion usually commends 
unconditional inversion in target position and recom- 
mends its denial for source position (cf. Eells & Fitelson, 
2002). Measures D, L, and Z share this pair of positive 
and negative properties. 

Conditional compositionality for conjunction: Lemma 3 (sec- 
tion 5.4) about L and LL exhibits the gold standard for 
useful conjunctive compositionality, in source position. 
The qualitative theorem 6 (section 3) presages its useful- 
ness. No other familiar measures quite match this. For 
truly stark contrast, consider Carnap’s C, which affords 
compositionality Rc(AB > H)p = Rc(A > H)p+Rc(B > A)p 
when P(A vB) = 1. This is neither practically useful nor 
intuitive. 
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Conditional compositionality for disjunction: Here C is very 
counterintuitive: Rc(A v B>H)p = Rc(A>H)p+ Rc(B> H)p 
when P(AB) = 0. In the case P(AB) = 0, intuitive, use- 
ful compositionality situates the relevance that source 
proposition AvB has for some target C between the 
disjuncts’ relevances. The Big Three and their isotonic 
transforms meet this requirement. The disjunction’s rel- 
evance is always some convex combination (i.e., a “mix- 
ture” or weighted mean) of the disjunct relevances. 


Theorem 10. 
(a) If P(AB) = 0, then Ry(A v B >C)p = aRr(A > C)p + 
(1 - a) Rx(B > C)p, where 0 sa <1 forT=D,Q 
and, if neither A nor B is extremely relevant to 
C, also for T = L. 


(b) The mixture weight ais ee for T = D, Q, 
P(A) + P(B) 
and it is BATES) for T =L. 


P(A| aC) + P(B | AC) 


The Big Three share this mixture property with the 
“expected utility” concept; see Savage (1954) and chap- 
ter 8.2 by Peterson (this handbook). This commonal- 
ity makes apt the characterization of each measure as a 
“quasi-utility,” a term that Good (1983), with a different 
motivation, had applied to LL. Quasi-utilities quantify 
the usefulness of “source” propositions toward promot- 
ing the “target” proposition in a credal context. Under 
this instrumental perspective, the relation between evi- 
dence and hypothesis is a special case of a more general 
rhetorical relationship. Establishment of a source propo- 
sition that is positively relevant to a target proposition is 
a resource toward the rhetorical end of establishing the 
target proposition. 


6. Applications in Psychology 


Besides uses in philosophy of science,’* there are psy- 
chological applications of relevance reasoning. The best 
known among them involve quantitative or merely ordi- 
nal judgment tasks and have a common feature. Exper- 
imental subjects appear, in some contexts, to answer 
requests for posterior probability or odds judgments with 
replies that are consistent with their being covert rel- 
evance judgments. Prominent instances of this are the 
phenomenon of “prior probability (or base rate) neglect” 
and the “conjunction fallacy.”"” 

The “neglect” phenomena, when they arise, may be 
seen as instances of Bayes’ Theorem in odds form (sec- 
tion 5.3). In this instance, prior odds for H are tacitly 
set to 1 (ie., assume equal probability of H and ~H). 
These odds fail to reflect information supplied on the 
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experimenter’s instruction sheet that should warrant a 
skewed prior, say, with P(H) > P(-H), before observation 
E impinges. Arithmetically, such information neglect 
will equate posterior odds with the likelihood ratio value 
for relevance. 

The conjunction fallacy arose most famously in the 
following example from the late 1970s. A fictitious per- 
son, Linda, was introduced on the experimental instruc- 
tion sheet with something like the following description, 
E: Linda, aged 31, is single, outspoken, and bright; has 
majored in philosophy; and has been demonstrating 
for equal opportunity causes. Participants were then 
requested to rank the probabilities of Linda being a bank 
teller (B), an active feminist (F), and an active feminist 
bank teller (BF)—presumably in the light of information 
E. A surprisingly large proportion of respondents ranked 
BF above B, which would contravene the probability law 
P(BF|E) s P(B| E). Now, E was designed to be “represen- 
tative” of F and “unrepresentative” of B. Many probabi- 
lists would explicate representativeness of a property ® 
by the relevance of statements such as “Linda is ®” to a 
reference description E, noting that F is positive to E and 
B negative. If one assumes that people responded to the 
experimenter’s request with covert relevance judgments, 
the seemingly anomalous ranking is entirely feasible 
and indeed natural. See Tentori, Crupi, and Russo (2013) 
and Cevolani and Crupi (2015) for leads on relevance 
solutions of the conjunction fallacy.” 

Finally, maximization of relevance (understood as 
increasing cognitive effects and reducing processing costs) 
has been conjectured in relevance theory to be a general 
principle structuring both cognition and communica- 
tion. Relevance theory is popular in linguistics (Wilson 
& Sperber, 2004) but differs from the accounts of eviden- 
tial relevance presented here. Empirically, this theory has 
been applied to the Wason selection task in Sperber, Cara, 
and Girotto (1995). Moreover, there is evidence that the 
measure delta-p = P(H|E)-P(H|-=E) (which is not dis- 
cussed above) serves as an explication of participants’ 
assessments of the relevance of E for H (see Skovgaard- 
Olsen, Singmann, & Klauer, 2016b). In Skovgaard-Olsen, 
Singmann, and Klauer (2016a), the relationship between 
this notion of relevance and indicative conditionals (“If 
A, then C”) was investigated. 


7. Conclusion 


Our discussion of qualitative relevance relations has 
emphasized the importance of conditional independence 
assumptions. In practice, this means that one should seek 
out such independencies in one’s domain of application. 


274 


Our discussion of relevance measures has shown the 
continued importance of conditional independence for 
several ordinal and compositionality properties of rel- 
evance. Another useful condition for compositionality, 
specifically of disjunctive evidence, was established for 
the major measures: the condition of mutual extreme 
negativity of disjuncts. More generally, our result on 
task-relative ordinal equivalence of the major measures 
provides an assurance that many ordinal relevance judg- 
ments are not unreasonably measure dependent. 

If one must quantify relevance, the log-likelihood 
ratio LL is the default choice. It is the weakest sufficient 
statistics and has useful conditional additivity for con- 
junctive evidence. The relevance quotient Q and its 
logarithm QL are useful, for one, when relevance direc- 
tionality is intrinsically unspecified. For special pur- 
poses, measures have been commended that may violate 
the desideratum POP-D discussed in section 5.4. Some 
philosophers indeed recommend measure pluralism to 
express diverse aspects of the relevance relation quanti- 
tatively. See Hajek and Joyce (2008) on both points. 


Notes 


Editorial note: Arthur Merin died on May 24, 2019, shortly after 
finishing this chapter. Final minor revisions were undertaken 
by the editors. 


1. See chapter 4.1 by Hajek and Staffel (this handbook). To a 
good extent, the considerations of this chapter could be carried 
out in terms of other representations of degrees of credence; 
see chapter 4.7 by Dubois and Prade and chapter 5.3 by Kern- 
Isberner, Skovgaard-Olsen, and Spohn (both in this handbook). 


2. The juxtaposition AB represents the conjunction of A and B. 


3. (A LC|+B)p can be weakened to (A not neg C|+B)p (W. E. 
Johnson; cf. Keynes, 1921). Theorem 5 follows from a result of 
Reichenbach (1956), which can be stated thus: If positive, neu- 
tral, and negative relevance polarities are represented by 1, 0, and 
-1, respectively, then, given (A 1 C | + B)p, the relevance polarity of 
A to C is the arithmetical product of the relevance polarities of A 
to Band of Bto C. 


4. The specific benefit of conditional independence is relevance- 
additivity under lemma 3, section 5.4. 


5. The language used in glosses is often counterfactual, par- 
ticularly when referring to the “likelihood,” P(D|H), of H in 
view of D as the probability that D “would have had,” given 
H. On likelihoods in statistics see, for example, Edwards (1972). 


6. The label “Transparency” stands for “no unavoidable sur- 
prises.” Hidden propositions and jumps would be surprises. 


7. Thus, Reo(E >H)p = tanh[ log. Ru(E > H)»], that is, KO is just 
a transform of LL. 
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8. This is Carnap’s 1950 name. Some authors call it “prob- 
ability ratio measure” and use the label R, which, however, has 
distinct earlier referents. 


9. A function f:R—R is isotonic iff a < b implies f(a) < f(b), for 
any a, bER. 


10. Reminder: O(H) =a_ P(H)/P(-H), P(A) = O(A)/[O(A) + 1], and 
O(H| E) =a P(H| E)/P(HH | E).- 


11. Crupi and Tentori label POP-D “Final Probability,” LIK-D 
“Law of Likelihood,” MCIE “Modularity for Conditionally 
Independent Data,” and MDHD “Disjunction of Alternative 
Hypotheses.” 


12. “Only if” parts for POP-D and LIK-D are in Steel (2003). 


13. For other measures validating POP-D, among them Z, see 
Crupi and Tentori (2016). A uniqueness result for Z is presented 
in Crupi and Tentori (2013). 


14. Unlike in statistics, the likelihood concept is extended by 
philosophers also to mutually compatible and nonexhaustive {Hj}. 


15. (3) is an inessential embellishment that makes things vivid. 


16. Applications of theorem 9 also include the generalization, 
to all measures validating POP-D, of purely ordinal results such 
as the following proposition (Merin, 1999): Let A V B=ar(A v B) a 
-(A vB) represent exclusive disjunction, and assume (A 1 B|+ H) 
and 1=4<R(A>H)p, R(B>H)p<o. Then 4 <R(AvB>H)p 
<max[Ri(A>H)p, Ri(B>H)p]< R(AB>H)p, and R(AVB>H)p 
<R(AvB>H)p, while none of R\(AVB>H)p 2 tu. are ruled 
out. The result shows, for one, that the defeasible “not both” 
implicature of assertions “A or B,” which is commonly held to 
be inferred from the nonassertion of “A and B,” can diminish 
the evidential relevance of inclusive “A or B” in issue-based 
discourse. 


17. This is closely connected to what Roberto Festa calls “Mat- 
thew properties,” referring to the biblical saying, “For onto every 
one that hath shall be given” (see Festa & Cevolani, 2016). 


18. For more detail on these, see the surveys, some of them 
extensive, by Huber (2007), by Hajek and Joyce (2008), Crupi 
(2015), and Crupi and Tentori (2016). 


19. See Kahneman (2011, chapters 15-16) on the two phenom- 
ena; see also chapter 1.2 by Evans and chapter 8.5 by Hertwig 
and Kozyreva (both in this handbook). 


20. Another application of evidential relevance (see, e.g., Merin, 
1999), which involves no cross-purposes in the interpretation of 
instructions, is to the theory of meaning for natural-language 
function words like “and,” “or,” “not,” “but,” “even,” “also,” 
“some,” and “many.” 
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4.4 Probability Logic 


Niki Pfeifer 


Summary 


This chapter presents probability logic as a rationality 
framework for human reasoning under uncertainty. 
Selected formal-normative aspects of probability logic 
are discussed in the light of experimental evidence. Spe- 
cifically, probability logic is characterized as a general- 
ization of bivalent truth-functional propositional logic 
(“classical logic,” for short), as being connexive, and as 
being nonmonotonic. The chapter discusses selected 
argument forms and associated uncertainty propagation 
rules. Throughout the chapter, the descriptive validity 
of probability logic is compared to classical logic, which 
was used as the gold standard of reference for assessing 
the rationality of human deductive reasoning in the 
20th century. 


1. Probability Logic Is a Generalization 
of Classical Logic 


Probability logic as a rationality framework combines 
probabilistic reasoning with logical rule-based reason- 
ing and studies formal properties of uncertain argument 
forms. 

Among various approaches to probability logic (for 
overviews, see, e.g., Adams, 1975, 1998; Coletti & Scozza- 
fava, 2002; Demey, Kooi, & Sack, 2019; Haenni, Romeijn, 
Wheeler, & Williamson, 2011; Hailperin, 1996), this 
chapter reviews selected formal-normative aspects of 
probability logic in the light of experimental evidence. 
The focus is on probability logic as a generalization of 
the classical propositional calculus (“classical logic,” for 
short; for probabilistic generalizations of quantified state- 
ments, see, e.g., Hailperin, 2011; Pfeifer & Sanfilippo, 
2017, 2019). The generalization is obtained (i) by the use 
of probability functions and (ii) by the introduction of 
the conditional event as a logical object, which is not 
expressible within classical logic. This generalization is 
currently most frequently investigated from a psychologi- 
cal point of view (see, e.g., Elqayam, Bonnefon, & Over, 


2016; Oaksford & Chater, 2010; Pfeifer & Kleiter, 2009), 
and it is thus most suitable for discussing both empirical 
and normative aspects of probability logic as a rationality 
framework. The empirical focus is on investigating gen- 
eral patterns of human reasoning under uncertainty and 
not on data modeling. 

Classical logic is bivalent as it deals with true and false 
as the two truth-values, which are assigned to proposi- 
tional variables (denoted by uppercase letters in italics; 
for classical logic, see also chapter 3.1. by Steinberger, 
this handbook). Truth tables can be used to define logi- 
cal connectives like conjunction (“A and B,” denoted by 
“A & B”), disjunction (“A or B,” denoted by “A v B”), nega- 
tion (“not-A,” denoted by “~A”), or the material condi- 
tional, which is the disjunction ~A v B. Probability logic 
generalizes classical logic by using the whole real-valued 
unit interval from 0 to 1 instead of just two truth-values: 
truth-value functions are generalized by probability 
functions. While classical logic is truth-functional, prob- 
ability logic is only partially truth-functional as, usu- 
ally, in the conclusion, even from point-valued premise 
probabilities, a probability interval is obtained. Prob- 
ability functions can be used to formalize a real or an 
ideal agent’s degrees of belief in propositions formed by 
logical connectives (e.g., in A&B, Av B, or ~A). More- 
over, conditional probability functions can measure the 
degree of belief in a conditional event, that is, p(B| A). The 
conditional event B|A is a three-valued logical entity 
that is true if A &B is true, false if A & ~B is true, and void 
(or undetermined) if ~A is true.’ Since the conditional 
event cannot be expressed by a two-valued proposition, 
it is by definition not propositional and constitutes a 
further generalization of classical logic. Using the bet- 
ting interpretation of probability, “true” means that the 
bet is won, “false” means that the bet is lost, and “void” 
means that the bet is called off (i.e., you get your money 
back). 

Note that p(A) = 0 does not imply that A is logically 
impossible (i.e., a logical contradiction L). However, p(L) 
is necessarily equal to zero. As it does not make sense to 
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add | to your stock of beliefs (or to bet on a conditional 
whose antecedent is L), it is obvious why “A | L” is unde- 
fined in the coherence approach.” The semantics of the 
conditional event matches the participants’ responses in 
the truth table task (see, e.g., Wason & Johnson-Laird, 
1972; see also chapter 4.6 by Oberauer & Pessach, this 
handbook): most people judge that (i) A&B confirms 
the conditional “If A, then B,” (ii) A& ~B disconfirms 
it, and (iii) ~A is irrelevant for “If A, then B.” Under the 
material-conditional interpretation, one would expect 
that rational people judge that ~A confirms the con- 
ditional. As this expectation was violated, the response 
pattern (i)-(iii) was pejoratively called the “defective 
truth table.” Within the rationality framework of prob- 
ability logic, however, this response is perfectly rational, 
as it matches the semantics of the conditional event 
(see, e.g., Kleiter, Fugard, & Pfeifer, 2018; Over & Barat- 
gin, 2017; Pfeifer & Tulkki, 2017b; see also chapter 6.2 by 
Over & Cruz, in this handbook). 


2. Conditional Probability, Zero-Probability 
Antecedents, and Paradoxes 


Traditionally, the conditional probability of B given A is 
defined by 


(1) p(B|A) =der p(A &B)/p(A), if p(A) > 0. 


Condition p(A) >0 here serves to avoid fractions over 
zero. But what if p(A) = 0? Then the conditional probabil- 
ity is undefined or default assumptions about p(B|A) are 
made. For example, some approaches suggest, by default, 
to equate p(B|A) with 1 in this case (e.g., Adams, 1998, 
p. 57, footnote 5). However, this leads to wrong results, 
since then also p(~B | A) = 1, which violates the basic prob- 
abilistic principle p(B | A) + p(~B | A) = 1 (for a discussion, 
see Gilio, 2002). Moreover, from a practical point of view, 
if p(B|A) is left undefined, counterintuitive consequences 
may follow. Consider, for example, the following “para- 
dox of the material conditional”: 


(2) B; therefore, if A, then B. 


Argument (2) consists of a premise B and a conditional 
as the conclusion. Under the material-conditional inter- 
pretation, (2) is logically valid (i.e., there is no model 
in which the premise set is true while the conclusion 
is false). However, natural-language instantiations can 
appear counterintuitive (substitute, for example, “The 
moon is made of green cheese” for A and “The sun will 
shine in Vienna” for B). This mismatch between the logi- 
cal validity of (2) and its counterintuitive instantiations 
constitutes the paradox. From a logical point of view, 
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a logically valid argument remains logically valid what- 
ever the instantiations are. If (2) is formalized in prob- 
ability logic, however, the paradox is blocked when the 
conditional is represented by a conditional probability; 
then the corresponding argument is probabilistically 
noninformative, that is, for all probability values x: 


(3) p(B) = x; therefore, 0 < p(B|A) < 1 is coherent for all 
probability values x. 


An argument is probabilistically noninformative if the 
premise probabilities do not constrain the probability 
of the conclusion. More technically, probabilistic non- 
informativeness means that for all coherent probability 
assessments of the premises, the tightest coherent prob- 
ability bounds on the conclusion coincide with the unit 
interval [0, 1]. 

What does “coherent” mean here? An assessment p 
on an arbitrary family K of conditional events is coher- 
ent if and only if, for any combination of bets on a finite 
subset of conditional events in K, it cannot happen that 
the values of the random gain,’ when at least one bet is 
not called off, are all positive or all negative (“no Dutch 
book”; see also chapter 4.1 by Hajek & Staffel, this hand- 
book). In the coherence-based approach, the avoidance 
of Dutch books is in the case of unconditional events 
equivalent to the solvability of a specific system of linear 
equations. This solvability reflects the existence of at least 
one probability distribution on a suitable partition of the 
constituents (i.e., the possible cases), which is compat- 
ible with the initial probability assessment. In geometrical 
terms, a probability assessment on n events can be rep- 
resented by a prevision point p (i.e., a vector in [0, 1]") 
and the set of constituents by a set Q of binary points 
(i.e., of vectors in {0,1}"). Then, p is coherent if and only 
if p belongs to the convex hull of Q (de Finetti, 1974). 
In the case of conditional events, coherence amounts to 
the solvability of a suitable finite sequence of systems of 
linear equations (see, e.g., Biazzo & Gilio, 2000; Coletti & 
Scozzafava, 2002). 

If the conditional in (2) is represented by the prob- 
ability of the material conditional, the paradox is inher- 
ited. For all probability values x: 


(4) p(B) = x; therefore, x < p(~Av B) < 1 is coherent. 


Probability values strictly less than x in the conclusion 
of (4) are of course not coherent. Note that in general, 
(3) is probabilistically noninformative for all positive 
premise probabilities (i.e., p(B) > 0). For the extreme case 
p(B) = 1, when conditional probabilities are defined by 
(1), then p(B|A) =1 or is undefined. This is obvious, 
since if p(B) = 1, then p(A &B) = p(A); therefore, by (1), 
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p(B|A) = p(A &B)/p(A) = p(A)/p(A) = 1, provided p(A) > 0. 
If p(A) = 0, then p(B|A) is undefined. This result is coun- 
terintuitive and does not match the experimental data: 
people interpret (3) as probabilistically noninformative, 
even in the case of p(B) = 1 (Pfeifer & Kleiter, 2011). How- 
ever, in coherence-based probability logic (see, e.g., Coletti & 
Scozzafava, 2002; Gilio, Pfeifer, & Sanfilippo, 2016), where 
p(B|A) is primitive and problems with zero-probability 
antecedents are avoided, 0 < p(B|A) < 1 is coherent even 
in the extreme case p(B) = 1 (for a detailed proof, see 
Pfeifer, 2014). This example shows that the evaluation 
of the rationality of a probabilistic inference depends on 
whether the underlying probability concept allows for 
dealing with zero-probability antecedents or not. In the 
framework of coherence, the probabilistic noninforma- 
tiveness of argument (3) holds for all probability values 
of the premises; however, for approaches that are based 
on (1), it holds only for positive probabilities. 


3. From the Truth Table Task to the Probabilistic 
Truth Table Task 


In the 20th century, the dominating rationality framework 
in the psychology of deductive reasoning was logic. Prom- 
inent examples are Braine and O’Brien’s “mental logic” 
(1998; see also chapter 3.2 by O’Brien, this handbook), 
the “mental rule theory” by Rips (1994), and Johnson- 
Laird’s (1983) theory of mental models (see chapter 2.3 
by Johnson-Laird, this handbook). The rationality frame- 
work of the former two theories is derived from classical 
logical proof theory (which is syntactic and “rule-based”), 
whereas the latter one is based on logical model theory 
(which is semantic and “model-based”). According to 
these logic-based rationality frameworks, people are ratio- 
nal if they use logically valid rules of inference (like modus 
ponens) or if they build mental models that are inspired 
by truth tables. With the advent of the “new paradigm 
psychology of reasoning,” which is characterized by using 
probabilistic rationality frameworks instead of logic, not 
only did the evaluation of the rationality of human infer- 
ence change but also the task paradigms were adapted. The 
abovementioned truth table task, for example, became a 
probabilistic truth table task (PTTT, for short) to investi- 
gate how people interpret conditionals (Evans, Handley, 
& Over, 2003; Oberauer & Wilhelm, 2003; see also chap- 
ter 4.6 by Oberauer & Pessach, this handbook). From a 
probability-logical point of view, the PTTT presented the 
following premises to the participants: 


(S) p(A &B) = x1, p(A&~B) = x2, p(~A &B) = x3, and 
p(~A &B) = Xa. 
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Then the participants were asked to infer their degree 
of belief in the conditional “If A, then B” based on the 
probabilistic information given in (5). The main experi- 
mental result obtained from this task was that most 
participants gave as their response values consistent 
with x1/(x%1+X2), which corresponds to the conditional- 
probability interpretation of the conditional (p(B|A); 
this is consistent with the “Ramsey test” as described in 
chapter 6.2 by Over & Cruz, this handbook). A signifi- 
cant minority responded with x1, which corresponds 
to the conjunction interpretation of the conditional 
(p(A &B)). Under the material-conditional interpreta- 
tion, one would expect x1+%3+X4 as the most frequent 
response type in this task. However, experimental evi- 
dence for this hypothesis was negligible. When the task 
was given several times to the same participants, among 
those who did not use conditional-probability responses 
in the first PITT tasks, a “shift of interpretation” was 
observed: over the course of the experiment, these par- 
ticipants “shifted” to the conditional-probability inter- 
pretation. In the last tasks of the experiment, about 80% 
of the responses were consistent with the conditional- 
probability interpretation. This indicates that condi- 
tional probability is a key building block for a rationality 
framework for human reasoning about conditionals 
under uncertainty (see Fugard, Pfeifer, Mayerhofer, & 
Kleiter, 2011; Pfeifer, 2013). 

In the PTTT, mostly indicative conditionals (“if- 
then” formulations) with “abstract” materials were used 
(like “If the figure shows a square, then the figure is red”). 
Interestingly, the finding that conditional probability 
is the best predictor for the data was also replicated for 
a larger variety of conditionals: causal conditionals (“If 
cause, then effect”), counterfactual conditionals (“If A 
were the case, then B would be the case”; Over, Hadjichris- 
tidis, Evans, Handley, & Sloman, 2007; Pfeifer & Stdckle- 
Schobel, 2015), and abductive conditionals (“If effect, 
then cause”; Pfeifer & Tulkki, 2017a). 

The next sections explain why probability logic vali- 
dates basic connexive principles and why it is nonmono- 
tonic under the conditional-probability interpretation 
of the conditional. 


4. Probability Logic ls Connexive 


Connexive logics are motivated by the idea that there 
should be some connection between antecedents and 
consequents of conditionals in the sense that they 
should not contradict each other. Connexive logics are 
alternatives to (classical) logic as they are neither con- 
tained in, nor proper extensions of, it (for an overview, 
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see Wansing, 2020). They include, for example, Aristo- 
tle’s theses: 


(AT1) ~ (if ~A, then A), 
and 
(AT2) ~ (if A, then ~A). 


Under the material-conditional interpretation, (AT1) 
and (AT2) are contingent in logic (i.e., (AT1) is logically 
equivalent to ~A and (AT2) is logically equivalent to A). 
Thus, within logic, it is rational to say that it depends on 
the truth-value of A whether (AT1) and (AT2) are true. 
Indeed, (AT1) and (AT2) are not tautologies in logic. 
Experimental data suggest, however, that people believe 
that (AT1) and (AT2) must be true (Pfeifer, 2012). Prob- 
ability logic allows for validating the rationality of (AT1) 
and (AT2). First, look at the conditionals (in terms of 
conditional probabilities): by coherence, p(A|~A) and 
p(~A | A) must be equal to zero. Second, the corresponding 
conditionals are negated by negating their consequents, 
that is, p(~A|~A) and p(~~A | A) (= p(A |A)), respectively. 
Since probability 1 is the only coherent assessment for 
p(~A|~A) and p(A|A), (AT1) and (AT2) are validated. 
This matches the experimental data (Pfeifer, 2012; 
Pfeifer & Tulkki, 2017b). 

Boethius’s theses are another instance of connexive 
principles. Like Aristotle’s theses, Boethius’s theses can 
be justified within probability logic (but not within clas- 
sical logic). Boethius’s theses are (the arrow denotes a 
conditional): 


(BT1) (A—~B)>~(A>~B) 
and 
(BT2) (A—-~B)>~(A—B). 


Under the narrow-scope negation interpretation 
of negating conditionals,* the antecedent of (BT1) is 
interpreted in probability logic by p(B|A) > x (for some 
threshold x >.5) and its consequent by p(~~B|A) > x. 
Since B is logically equivalent to ~~B, (BT1) holds in 
probability logic. Analogously, (BT2) is validated in 
probability logic. Under the material-conditional inter- 
pretation, neither (BT1) nor (BT2) hold in general: (BT1) 
and (BT2) are both logically equivalent to A. Moreover, 
Abelard’s first principle, which is another connexive 
principle, can be rationally justified in probability logic: 


(AFP) ~((A—>B) & (A>~B)). 


Since, in general, p(B|A) + p(~B|A) = 1, it cannot be 
the case that both p(B|A) and p(~B|A) are “high” 
(i.e., at least strictly greater than .5). Therefore, (AFP) is 
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validated in probability logic. However, under the material- 
conditional interpretation, (AFP) is logically equivalent 
to A. Aristotle’s and Boethius’s theses and Abelard’s first 
principle are intuitively plausible principles that hold in 
connexive and in probability logic but not in classical 
logic. Aristotle’s theses have received strong experimen- 
tal support (Pfeifer, 2012; Pfeifer & Tulkki, 2017b). For 
the other connexive principles, future empirical research 
is needed. 


5. Probability Logic ls Nonmonotonic 


Nonmonotonic reasoning is about retracting conclu- 
sions in the light of new evidence. For example, from 
“Tf this animal is a bird (B), then this animal can fly (F),” 
one would not want to conclude “If this animal is a bird 
and a penguin (B&P), then this animal can fly (F).” 
Classical logic, however, is monotonic: adding premises 
to a logically valid argument can only lead to an increase 
but never to a decrease of the conclusion set (for an 
overview, see, e.g., Antoniou, 1997; Strasser & Antonelli, 
2019). Therefore, conclusions cannot be retracted in 
classical logic. Under the material-conditional interpre- 
tation, the abovementioned argument is logically valid. 
The argument form is called “monotonicity” (or “prem- 
ise strengthening”): 


(MON) ~BvF logically implies ~(B & P) v F. 


In probability logic, however, the corresponding argu- 
ment form is probabilistically noninformative, and 
monotonicity is therefore blocked, that is, for all prob- 
ability values x: 


(6) p(F|B) = x; therefore, 0 < p(F|(B&P)) <= 1 is coherent. 


Basic rationality principles for retracting conclusions 
in the light of new evidence are concentrated around Sys- 
tem P (Kraus, Lehmann, & Magidor, 1990). The principles 
of System P are broadly considered to be minimal ratio- 
nality requirements for any system of nonmonotonic 
reasoning. It is therefore a key system for reasoning in 
general. Various different semantics were developed for 
nonmonotonic reasoning systems, some of which are 
probability-logical (see, e.g., Adams, 1975; Gilio, 2002; 
Goldszmidt & Pearl, 1996; Hawthorne & Makinson, 
2007; Schurz, 1997). Psychologically, Pfeifer and Kleiter 
(2005, 2006b, 2010) and Pfeifer and Tulkki (2017b) pre- 
sent experimental data supporting the descriptive validity 
of the coherence-based probability semantics of System P 
(Gilio, 2002; for experimental studies on the possibilistic 
semantics of System P, see, e.g., Benferhat, Bonnefon, 
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& Da Silva Neves, 2005; Da Silva Neves, Bonnefon, & 
Raufaste, 2002). In the coherence semantics, default con- 
ditionals like “If A, then normally B”> are interpreted as 
“high” coherent conditional probabilities that may also 
be imprecise (e.g., interval-valued probabilities). For each 
rule of System P, Gilio (2002) proved the probability prop- 
agation rules, which describe how the probabilities of the 
premises are propagated to the conclusion. As an exam- 
ple, consider the and-rule. For all probability intervals 
[X1, Xu] and [y1, vu]: 


(AND) x. <p(B|A) < xu and y, < p(C|A) s yu; therefore, 
max{0, x1+¥i- 1} s p((B&C)|A) s min{xy, yu} is 


coherent. 


It can easily be seen that even in cases where the 
premise probabilities are point-valued (i.e., x. = xy and 
¥Vi = yu), the probability of the conclusion is usually 
interval-valued. In the extreme case where the premise 
probabilities are equal to 1, the only coherent conclu- 
sion probability is also equal to 1. 

Experimental data suggest that most people infer coher- 
ent interval responses in inference tasks corresponding to 
(AND). The majority of those people who violate coher- 
ence violate the lower bound (Pfeifer & Kleiter, 2005). 
These data speak also against the common misunder- 
standing that people are unable to perform probabilistic 
reasoning because of the high frequency of “conjunction 
fallacies” allegedly committed in Tversky and Kahne- 
man’s (1983) well-known Linda task. The conjunction 
fallacy consists in ranking the probability of a conjunc- 
tion (B&C) as higher than that of one of its conjuncts (C). 
In the context of (AND), this would mean that the upper 
probability bound on the conclusion is violated. In the 
experimental data on (AND), however, the people who 
violated the coherent interval violated the lower probabil- 
ity bound (Pfeifer & Kleiter, 2005). Even if the terms “and” 
and “probability” are mentioned, that does not mean 
that actual conjunction probabilities are investigated in 
the Linda task: the participants might not interpret the 
task as a task about conjunction probabilities. 

Concerning the other rules of System P, a strong agree- 
ment between the participants’ interval responses and 
the coherent intervals was observed (Pfeifer & Kleiter, 
2005, 2006b, 2010; Pfeifer & Tulkki, 2017b). Moreover, 
the majority of the participants correctly understood 
that (6) and that contraposition (e.g., for all probability 
values x: if p(B| A) = x, then 0 < p(~A|~B) < 1 is coherent) 
are probabilistically noninformative. This is interesting 
as these argument forms are logically valid under the 
material-conditional interpretation, but they cannot be 
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validated without further assumptions in a nonmono- 
tonic reasoning system. Adding monotonicity or contra- 
position to System P, for example, would make System 
P monotonic (which is undesirable of course). Transitiv- 
ity is also probabilistically noninformative (i.e., for all 
probability values x and y: if p(B|A) =x and p(C|B) =y, 
then 0 < p(C|A) <1 is coherent), and its addition to 
System P would make it monotonic. Experimental data 
suggest that people interpret the task material of Transi- 
tivity (presumably for pragmatic reasons) as Cumulative 
Transitivity (CUT) of System P (Pfeifer & Kleiter, 2006b, 
2010): CUT changes the premises of Transitivity by con- 
junctively adding the antecedent of the first premise to 
the antecedent of the second premise. For all probabil- 
ity values x and y: 


(CUT)  p(B|A) = x, p(C|(A &B)) = y; therefore, 


xy < p(C|A) s xy+1-x is coherent (Gilio, 2002). 


Note that the probability propagation rules of (CUT) 
coincide with those of the probabilistic modus ponens 
(Pfeifer & Kleiter, 2006a); for all probability values x and y: 


(MP) p(B) =x, p(C|B) =y; therefore, xy < p(C) sxy+1-x 


is coherent. 


This close relationship between (CUT) and (MP) is 
explained by the fact that unconditional probabilities are 
defined in probability logic by the following principle: 


(7) p(A) =aer, p(A|T), where “T” denotes a logical tautology. 


By replacing A with Tin (CUT) and by (7), we obtain 
(MP). Modus ponens is one of the most frequently inves- 
tigated argument forms in the psychology of reasoning. 
Its nonprobabilistic version is usually endorsed by most 
participants (Evans, Newstead, & Byrne, 1993). The clear 
majority of responses in tasks on probabilistic modus 
ponens support the predictions by probability logic 
(Pfeifer & Kleiter, 2009; Pfeifer & Tulkki, 2017b). 


6. Concluding Remarks 


This chapter characterized probability logic as a gener- 
alization of logic and explained the importance of zero- 
probability antecedents. Probability logic is connexive 
and nonmonotonic. It is a powerful tool for investigating 
the rationality of reasoning in a unified and systematic 
way. Experimental studies support its descriptive validity. 
Future normative and descriptive research should include 
nested and compound conditionals. Probabilistic modus 
ponens and other argument forms, for example, were 
recently generalized to deal with nested conditionals and 
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compounds of conditionals (Gilio, Pfeifer, & Sanfilippo, 
2020; Sanfilippo, Pfeifer, & Gilio, 2017; Sanfilippo, Pfeifer, 
Over, & Gilio, 2018). Interestingly, the uncertainty propa- 
gation rules for modus ponens involving nested condition- 
als coincide with those of nonnested modus ponens (MP). 
For instance, consider the following nested instance of 
modus ponens: from “The cup breaks if dropped” (DB) 
and “If the cup breaks if dropped, then the cup is fragile” 
((D—B) —F) infer “The cup is fragile” (F). Here, the lower 
bound on the degree of belief in the conclusion F equals 
the product of the degrees of belief in the premises, and 
the upper bound on F equals the sum of the lower bound 
on F plus 1 minus the degree of belief in DB, which 
coincides with the uncertainty propagation rules of the 
nonnested (MP) (for details, see Sanfilippo et al., 2017). In 
this approach, which exploits the notions of conditional 
random quantities and conditional previsions, the law of 
import-export does not hold (Gilio & Sanfilippo, 2014; 
Sanfilippo, Gilio, Over, & Pfeifer, 2020), which is key to 
blocking Lewis’s (1976) notorious triviality results. Lewis’s 
triviality results show that sentences like (DB) — F must 
not be simply interpreted by p(F| (B|D)). Rather, for prop- 
erly investigating such structures, a richer formal structure 
is required. Future work is needed to assess the psychologi- 
cal plausibility of this approach. 

Finally, one might wonder why various nonclassical 
logics—which validate intuitively plausible rational- 
ity principles—were broadly neglected in the psycho- 
logical literature, even if they were available already for 
decades. The reasons might be due to research traditions. 
This chapter proposed probability logic as a normatively 
and descriptively appealing rationality framework for 
human reasoning, which combines (i) the requirement 
of plausible qualitative logical principles (like nonmono- 
tonicity and connexivity) with (ii) the expressibility of 
quantitative degrees of beliefs for investigating reasoning 
and argumentation under uncertainty. 
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Notes 
1. Note that the conditional event must not be nested: nei- 


ther A nor B in B|A may contain occurrences of “|”, because 
of Lewis’s (1976) triviality results. For nested conditionals and 
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logical operations among conditionals, more complex structures 
are needed to avoid triviality, which go beyond the scope of this 
chapter (see, e.g., the theory of “conditional random quanti- 
ties”; Gilio, Pfeifer, & Sanfilippo, 2020; Gilio & Sanfilippo, 2014; 
Sanfilippo, Gilio, Over, & Pfeifer, 2020; Sanfilippo, Pfeifer, & 
Gilio, 2017; Sanfilippo, Pfeifer, Over, & Gilio, 2018). 


2. Popper functions, however, allow for conditioning on 
contradictions (for a discussion, see Coletti, Scozzafava, & Van- 
taggi, 2001). 


3. In betting terms, you evaluate the probability p of an event 
E on the understanding that, for each real number s, you are 
willing to pay ps with the proviso that you will receive either 
s or O according to whether E happens or does not happen, 
respectively. The random gain (or “net gain”) G is the dif- 
ference between what you receive and what you pay. Thus, 
G =s-ps, when E is true, and G=-ps, when E is false. For a 
probability assessment on a family of events F, the associated 
random gain is the sum of the random gains of each bet on all 
events in F. 


4. For a wide-scope negation interpretation of negating con- 
ditionals in probability logic, see Gilio, Pfeifer, and Sanfilippo 
(2016) and Pfeifer and Sanfilippo (2017). 


5. Here, the scope of “normally” is the whole conditional and 
not just the consequent B. In other words, this means “The 
conditional ‘If A, then B’ holds by default” or “Normally, B fol- 
lows from A.” 
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4.5 Bayesian Rationality in the Psychology of Reasoning 


Nick Chater and Mike Oaksford 


Summary 


Bayesian models have become widespread throughout 
the cognitive and brain sciences, as providing natural 
ways to understand how the brain deals with uncer- 
tainty in perception, belief updating, decision making, 
and motor control. In this chapter, we ask how far the 
Bayesian approach helps psychologists understand per- 
haps the most direct expression of human rationality: 
explicit verbal reasoning. Historically, verbal reasoning 
has been modeled in the framework of mathematical 
logic; the “new paradigm” in the psychology of reason- 
ing developed over the past 25 years has suggested that 
human reasoning and argumentation may often be bet- 
ter understood using probabilistic Bayesian methods. 


1. The Psychology of Verbal Reasoning: 
A Brief History 


Following in the footsteps of 20th-century analytic phi- 
losophy, the starting point for the psychology of verbal 
reasoning in the 1960s was the assumption that the stan- 
dards of formal, binary logic should judge human rea- 
soning.’ Accordingly, researchers focused on how people 
reason with the so-called logical terms of language, most 
notably those that have natural counterparts in first- 
order logic: and, or, not, if-then, all, some. In a typical 
experiment, people would be given one or more prem- 
ises (e.g., Birds fly, Tweety is a bird) and asked to generate, 
or evaluate, possible conclusions that may follow (e.g., 
Tweety flies). The assessment of specific patterns of logi- 
cal reasoning was seen as central to a much wider project 
within cognitive science and artificial intelligence: view- 
ing cognition as a whole as operating through logical 
reasoning over world knowledge encoded in an internal, 
logical, language of thought—a program encapsulated 


(1) If I turn the hot tap, I get hot water 


I turn the hot tap 


Therefore: 1 get hot water 


in the phrase “cognition is proof theory” (Fodor & 
Pylyshyn, 1988). 

Since the mid-1990s, however, the psychology of 
verbal reasoning has begun to adopt what has become 
known as the “new paradigm” (Over, 2009), in which 
Bayesian probability theory, rather than logic, has been 
taken as the standard against which human perfor- 
mance should be compared (Oaksford & Chater, 1994). 
Moreover, the shift to a Bayesian viewpoint has also 
been associated with two connected changes: a focus on 
knowledge-rich real-world inferences, rather than nar- 
row logical puzzles, and viewing verbal reasoning not as 
an isolated, internal cognitive process but as having an 
inherently social function, in argument and persuasion. 

We begin with a brief sketch of logic-based accounts 
of reasoning and its difficulties before turning to proba- 
bilistic accounts of deduction, induction, abduction, and 
argumentation, concluding with brief reflections on how 
Bayesian approaches to verbal reasoning may connect 
with the wider program of Bayesian cognitive science.” 


2. Logic-Based Approaches to Reasoning 


In line with early research in the psychology of reason- 
ing, let us take logic as our starting point and consider 
the modus ponens (MP) inference (see sentence display 
1 below). 

In classical logic, this inference is of course valid: the 
truth of the premises guarantees the truth of the con- 
clusion, and it can readily be converted into an experi- 
ment: participants are instructed to assume the premises 
are true (whether they believe them or not) and to draw 
inferences from those premises alone. Notice that, from 
a Bayesian point of view, such reasoning may be highly 
unnatural: our subjective probabilities will, after all, typ- 
ically be influenced by the entirety of our background 


If p then g (A) 
Pp (B) 
q (MP: A, B) 
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knowledge—we shall return to the implications of this 
observation below. 

According to logic-based psychological approaches, 
verbal reasoning follows logical lines: either directly, 
through implementing a psychological version of the 
logician’s proof theory (a putative mental logic; Rips, 
1994; see chapter 3.3 by Khemlani, in this handbook), 
or indirectly, through implementing a psychological 
version of the logician’s model theory (mental model 
theory [MMT], Johnson-Laird, 1983; see chapter 2.3 by 
Johnson-Laird, this handbook, and, for a recent critique 
of MMT, Oaksford, Over, & Cruz, 2019). But the appro- 
priateness of any logic-based approach is thrown into 
question by the following observation. Suppose a friend 
says I didn’t get hot water this morning; by the logical law 
of modus tollens (MT), we can infer You didn’t turn the 
hot tap. But this is precisely the opposite of the correct 
conclusion: our friend is telling us that she didn’t get 
hot water despite turning the hot tap (otherwise, the non- 
appearance of hot water would be too uninteresting to 
mention). So, in real discourse, we confidently infer that 
the hot tap was turned, in precise contradiction to the 
apparent recommendations of logic. 

To explain how we reason in such cases requires rec- 
ognizing, presumably using general principles of con- 
versation (e.g., Grice, 1975; Sperber & Wilson, 1986), 
that the speaker is aiming to be relevant and informa- 
tive. Moreover, what counts as relevant and informative 
depends not just on the given premises but on back- 
ground knowledge (e.g., that a lack of hot water is suf- 
ficiently bothersome to be worth mentioning). 

Note, too, that this example illustrates how real-world 
inferences routinely violate a fundamental law of classi- 
cal treatments of the conditional. According to Strength- 
ening of the Antecedent, If p then q implies If p and r then 
q for any r. Yet If I turn the hot tap, I get hot water does 
not imply If I turn the hot tap and the heating is broken, I 
get hot water. Indeed, quite the opposite! Unlike logical 
reasoning, everyday reasoning is nonmonotonic: conclu- 
sions can be overturned if more information is added 
(e.g., Oaksford & Chater, 1991). 


3. Deductive Reasoning with Propositions 


Patterns of reasoning about the logical connectives, and, 
or, not, and, in particular, ifthen, have been intensively 
studied experimentally and theoretically. One approach 
is probability logic (see chapter 4.4 by Pfeifer, this 
handbook), which seeks to generalize standard logic by 
requiring that reasoners respect not merely logical con- 
sistency but also probabilistic coherence: degrees of belief 
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associated with propositions should be consistent with 
probability theory (Coletti & Scozzafava, 2002; Pfeifer & 
Kleiter, 2009). Moreover, researchers in the new paradigm 
have explored a variety of related formal ideas ranging 
from so-called p-validity, Bayesian and Jeffrey condition- 
alization, and dynamic inference. 

We begin by illustrating how probabilistic coherence 
applies to a generalization of the MP inference illus- 
trated in (1), above: 


(2) Ifptheng Pr(q|p)=a 


p Pr(p) = b 


Therefore: q Pr(q) = [ab,ab+(1-b)] 


Now, we no longer suppose that the premises are defi- 
nitely true. Instead, each premise is assigned a proba- 
bility (a and b, respectively), reflecting their degree of 
belief. Probabilistic coherence demands that the degree 
of belief in the conclusion must lie in the interval 
between (and including) the values ab and ab+(1-b).? 

What exactly does it mean to talk about the probabil- 
ity of a conditional Pr(if p then q)? The simplest approach 
is to equate this with the conditional probability Pr(q|p), 
an identification known as “the Equation” (Edgington, 
1995). The conditional probability itself will depend on 
arbitrary background knowledge, using what is known as 
the Ramsey test, proposed by philosopher and mathema- 
tician Frank Ramsey (1929/1990). We add the anteced- 
ent (e.g., The hot tap was turned) of the conditional to our 
entire set of beliefs, adjust them to take account of this 
new information, and read off the probability of the con- 
clusion (There is hot water) from this new set of beliefs. 
Notice, in particular, that from this point of view, reason- 
ing depends not just on the given premises but on world 
knowledge in general. 

Is the Equation appropriate from a psychological 
point of view? It has been confirmed in a large num- 
ber of experiments (e.g., Evans, Handley, & Over, 2003; 
Oberauer & Wilhelm, 2003; Politzer, Over, & Baratgin, 
2010), and experimental participants reject a crucial step* 
in philosophical arguments (Lewis, 1976) that appear to 
undermine the Equation by suggesting that it leads to an 
unacceptable triviality result (Douven & Verbrugge, 2013). 
Notice, too, that a conventional material-implication 
interpretation of the conditional (that Ifp then q is equiva- 
lent to Not-p or q) makes very different predictions, which 
are strongly at odds with participants’ judgments. 

How does probability coherence fare in explaining 
empirical data about how people reason with condition- 
als? Varying a and b, in the MP inference above, is seen 
reliably to yield probabilities of the conclusion that fall 
within the predicted coherence interval at above-chance 
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levels (Cruz, Baratgin, Oaksford, & Over, 2015; Evans, 
Thompson, & Over, 2015; Pfeifer & Kleiter, 2009; Politzer 
& Baratgin, 2016; Singmann, Klauer, & Over, 2014). This 
is not true for all inference types, however, including 
modus tollens (if p then g; not-g; therefore not-p). Cruz 
et al. (2015) argue that this may stem from the complex- 
ity of the inference and show that probabilistic coherence 
is respected above chance for eight simple one-premise 
arguments (e.g., p or q; therefore if not-p then q). 

Just as logical consistency has a parallel in probabi- 
listic coherence, so logical validity has a corresponding 
probabilistic analogue, p-validity (Adams, 1998; Coletti & 
Scozzafava, 2002; Pfeifer & Kleiter, 2009). A p-valid argu- 
ment is one for which it is not probabilistically coherent 
that the uncertainty of the conclusion (1 - Pr(conclusion)) 
exceeds the sum of the uncertainties of the premises 
(/,(1 — Pr( premise;))). 

For example, let us reconsider (2). Suppose, for exam- 
ple, a = .9 and b = .5, so that the sum of the premise uncer- 
tainties is .1+.5 = .6. According to probabilistic coherence, 
the probability of the conclusion is in the interval [.45, 
.95], with corresponding uncertainties, therefore, rang- 
ing from .55 down to .05. So it is not probabilistically 
coherent that the uncertainty of the conclusion (in the 
interval [.05, .55]) exceeds the sum of the premise uncer- 
tainties, .6. Hence, the inference is p-valid (and, indeed, 
this will be true for MP quite generally). 

P-validity mitigates all the more puzzling paradoxes 
that arise from the “material-conditional” (or what logi- 
cians prefer to call “material-implication”) interpreta- 
tion of If p then q used in standard first-order logic: that, 
as noted above, is equivalent to Not-p or qg. According to 
this interpretation, numerous apparent paradoxes arise 
(see chapter 5.2 by Rott and chapter 6.1 by Starr, both in 
this handbook). For example, the premise There is no life 
on Mars appears to imply If there is life on Mars, there are 
kangaroos on the moon. Such inferences are not, though, 
p-valid (and, indeed, people do not endorse them; see 
Pfeifer & Kleiter, 2011). 

Despite its theoretical interest, however, p-validity 
does not appear directly to conform to human reason- 
ing judgments. Indeed, from a psychological point of 
view, reasoning seems oriented to determining how one 
should change one’s beliefs in the light of new evidence 
and background knowledge (Harman, 1986), rather than 
determining the validity of arguments, whether validity 
is viewed from a logical or a probabilistic standpoint. 

So far, we have considered the situation in which 
both premises are potentially uncertain. But what hap- 
pens if one premise is learned to be true for certain? For 
example, suppose I am certain that you turned the hot 
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tap, because I saw you do it or because I know you to be 
completely truthful. Then, Pr(p) = 1 in (8), and the prob- 
ability of the conclusion can directly be read off from 
the conditional probability of the conclusion, q, given p: 
Pr(q| p). This is an example of what is known as Bayes- 
ian conditionalization, and it yields a precise probability 
rather than an interval. This approach (combined with 
background assumptions concerning the probability 
of q given not-p) successfully captured early empirical 
data (Oaksford & Chater, 2007; Oaksford, Chater, & Lar- 
kin, 2000) on how people reason with MP, MT, and two 
logically invalid patterns of reasoning with conditionals 
(Denying the Antecedent [DA], where not-p is taken to 
imply not-q, and Affirming the Consequent [AC], where 
q is taken to imply p). 

But suppose we are not completely certain that the 
hot water tap is turned on (i.e., what if Pr(p) < 1)? A natu- 
ral assumption from prior knowledge is that most taps, 
most of the time, are not being turned: tap-turning events 
are rare. So given evidence that the tap was turned may 
considerably raise an initial estimate, say, Pro(p) = .01, to 
Pri(p) = .9. In such cases, Jeffrey conditionalization (Jef- 
frey, 2004) provides a more general updating procedure 
(see chapter 4.1 by Hajek & Staffel, this handbook): 


(10) Pri(q) = Pro(q| p)Pri(p) + Pro(q| =p)(1- Pri(p)). 


Here, then, the probability of the conclusion Pri(q) will 
depend on prior assumptions about Pro(q| =p), so that 
belief updating depends on background knowledge even 
for modus ponens. Zhao and Osherson (2010; see also 
Hadjichristidis, Sloman, & Over, 2014) asked people to 
estimate the relevant probabilities and found that the 
probability of the conclusion aligns well with Jeffrey’s 
tule. 

We have so far assumed that changing the probabil- 
ity of one premise does not modify the probabilities of 
others. But this will not, of course, always be true, and 
in such cases, known as dynamic inference, reasoning is 
less straightforward (Adams, 1998; Gilio & Over, 2012; 
Oaksford & Chater, 2007, 2013). 

Recall the example we outlined earlier, where we are 
informed You didn’t get hot water this morning, but learn- 
ing this will drastically change our belief in the condi- 
tional premise If you turn the hot tap, you get hot water. 
If it were not revised, we would simply conclude You 
didn’t turn the hot tap. But the appropriate conclusion 
is that the probability of the conditional should drasti- 
cally be revised downward: while heating systems nor- 
mally work, learning You didn’t get hot water this morning 
leads to the strong suspicion that your heating system is 
malfunctioning. 
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Such cases, where other premises modify, perhaps 
drastically, the conditional probability of the conditional 
premise, are said to violate invariance or rigidity between 
the initial (Pro) and the revised distribution (Pr)): such 
violations imply that Pri(q|p) # Pro(q|p). They are some- 
times, but not always, observed (Zhao & Osherson, 2010), 
and allowing violations for MT (as in our example), as 
well as for AC and DA, improves fits to empirical data on 
conditional reasoning (Oaksford & Chater, 2007, 2013). 


4. Deductive Reasoning with Quantifiers 


So far, we have considered how people reason about 
combinations of, and relations between, propositions p, 
q, and so on. Researchers have also studied how people 
reason with the quantifiers all, some, some-not, and none, 
as well as with the so-called generalized quantifiers, most 
and few. For example, consider the syllogism 


(3) All beekeepers (B) are artists (A) 
Some chemists (C) are beekeepers 


Therefore: Some chemists are artists 


How can quantified inference be viewed in probabilistic 
terms? The Probability Heuristics Model (PHM; Chater & 
Oaksford, 1999) takes a direct approach, viewing quanti- 
fied assertions as expressing probabilistic constraints: All: 
Pr(y|x) = 1, Some: Pr(y | x) > 0, Some-not: Pr(y | x) < 1, None: 
Pr(y|x) = 0, Few: 0<Pr(y|x)<A, Most: 1-A<Pr(y|x) <1, 
for small A. 

How these probabilities combine depends on the struc- 
tural relationships between the premises—what is known 
in syllogistic logic as “figure” (i.e., the relative positions 
of the end terms, A and C, with respect to the middle 
term that links them, B). Each syllogistic figure can be 
represented by a dependency graph. For example, (3) has 
the following structure: Chemists — beekeepers — artists. 
The premises constrain the probabilities associated with 
the links in the graph. Here, for example, Some chemists 
are beekeepers, which applies to the first link in the graph 
and embodies the constraint that Pr(beekeeper | chem- 
ist) > O. Similarly, turning to the second link, All beekeep- 
ers are artists, corresponds to Pr(artist | beekeeper) = 1. It is 
then possible to apply the notion of p-validity and to 
prove whether the conclusion Some chemists are artists 
follows (which in this case it does). 

PHM does not predict reasoning behavior directly from 
p-validity, however (indeed, as we noted above, it seems 
that validity, whether probabilistic or logical, is not com- 
puted by reasoners, who are concerned instead with belief 
updating). Instead, PHM applies simple heuristics that 
reflect p-validity indirectly. The min-heuristicis defined over 
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an “informativeness” ordering of the various quantified 
statements, such that all> most > few > some > none > some- 
not (based on Shannon surprisal: I(s) = 1/Pr(s); Shannon & 
Weaver, 1949). This ordering is justified by assuming rar- 
ity: most predicates apply only to small subsets of enti- 
ties (i.e., there are far more nonartists than artists). The 
min-heuristic is that the quantifier in the conclusion 
should match the least informative quantifier in either 
of the premises. Thus, in (3), the conclusion should use 
the quantifier some. The max-heuristic is that the degree 
of confidence in the conclusion should depend on the 
expected informativeness of the quantifier in the most 
informative premises (here, all). Expected informative- 
ness is ordered: all > most > few > some > some-not ~ none, 
where the expectation is taken over the conclusions of the 
p-valid syllogisms for which that quantifier is the most 
informative premise (so p-validity enters indirectly into 
the analysis at this point). 

PHM fits prior experiments using logical quantifiers 
that have been modeled using mental logic (Rips, 1994), 
with fewer parameters (Chater & Oaksford, 1999). More- 
over, extending the approach to syllogisms with most 
and few led to data that corroborate the PHM account; 
these data remain outside the scope of any current logic- 
based account (Chater & Oaksford, 1999). Mental model 
theory (Johnson-Laird, 1983) provides an equally good 
account of experimental results that link memory span 
to syllogistic reasoning (Copeland & Radvansky, 2004), 
but PHM seems better able to capture data for syllogisms 
with many premises (Copeland, 2006). The patterns in 
syllogistic reasoning are complex and remain contested; 
one model comparison suggested that no current model 
of syllogistic reasoning is fully satisfactory (Khemlani & 
Johnson-Laird, 2012). 

One promising recent approach is the Probabilistic 
Representation Model (PRM; Hattori, 2016), which com- 
bines aspects of PHM and mental model theory. PRM 
proposes that people construct, and reason with, a prob- 
abilistic prototype model, capturing the eight joint prob- 
abilities ranging over the terms, A, B, and C, and their 
negations (e.g., Pr(A, ~B, C)). PRM captures these eight 
probabilities using a simple probabilistic model. The 
premises may also imply that some of the joint probabil- 
ities will be zero. PRM then draws a random sample from 
the distribution, which will thereby automatically be 
consistent with the premises. For example, PRM might 
sample a person who is an artist and a chemist but not a 
beekeeper. A sequential application of the min-heuristic 
then determines the conclusion that is consistent with 
the sample. As in PHM, conclusion type is determined 
by the min-heuristic. In (3), Some C are A is a possible 
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conclusion, if the sample contains at least one person 
who is both a chemist and an artist. If not, PRM checks 
for a conclusion with the next most informative quanti- 
fier and so on iteratively. If no conclusion is consistent 
with the sample, PRM generates “no valid conclusion.” 
As in PHM, confidence in a conclusion is determined by 
the max-heuristic. 

PRM models the empirical data slightly better than 
either PHM or mental model theory; it is also interesting 
that the best-fit sample size is six to seven items, roughly 
consistent with typical assumptions about working- 
memory capacity. PRM is also theoretically appealing, in 
that it assumes that reason operates through sampling 
from a probabilistic model, in a similar way to sam- 
pling models in other areas of cognition (e.g., Sanborn 
& Chater, 2016; Stewart, Chater, & Brown, 2006; Vul, 
Goodman, Griffiths, & Tenenbaum, 2014). 


5. Inductive Reasoning 


The psychology of verbal reasoning initially focused on 
deduction. Yet, from a Bayesian standpoint, reasoning is 
typically uncertain and knowledge-rich. Thus, Bayesian 
models are particularly well placed to deal with inductive 
verbal reasoning, where conclusions may extend, often 
very substantially, beyond the logical consequences of 
the premises. Inductive verbal reasoning also provides 
a bridge to the application of Bayesian models in per- 
ception, categorization, learning, and so on, which are 
widespread throughout the cognitive and brain sciences. 

One of the best-studied areas of inductive verbal rea- 
soning concerns property induction, such as 


(4) Blackbirds have property X 


Therefore: _Penguins/worms have property X 


Such an induction cannot be a matter of pure logic. After 
all, any such inference will be logically invalid, as it is 
perfectly possible that the premise is true and the con- 
clusion false. But also the difference between inferences 
concerning penguins and worms does not depend on logi- 
cal terms but on the nature of the properties themselves. 
So, for example, the fact that penguins generally seem 
more similar to blackbirds than either is to worms helps 
explain why people often generalize properties from 
blackbirds to penguins more readily (e.g., concerning 
their anatomy and physiology). 

Yet the picture is more complex—and crucially 
depends on the nature of the property and relevant 
background knowledge. So, for example, if the prop- 
erty is being contaminated by a rare poison, then people 
may suspect that worms and blackbirds may share this 
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property and that blackbirds may ingest the poison by 
eating worms or through living in the same environ- 
ment, while penguins live in a completely different eco- 
system (e.g., Kemp & Tenenbaum, 2009). 

A further natural question concerns the impact of 
multiple premises. Consider the following example: 


(S) Blackbirds have property X 
Ostriches have property X 


Therefore: Penguins have property X 


Here, the diversity of examples seems persuasive (e.g., 
Heit & Feeney, 2005). If blackbirds and sparrows have 
property X, we may suspect it is limited to garden birds; 
if blackbirds and ostriches do, we may suspect X is very 
widespread across birds as a whole, most likely includ- 
ing penguins. Getting such predictions right requires 
capturing complex knowledge, which may, for example, 
be modeled by hierarchical Bayesian models (Kemp & 
Tenenbaum, 2009). 

Abductive reasoning involves “inference to the best 
explanation” given a set of sensory, linguistic, or scien- 
tific data (Harman, 1965). But how do we decide which 
explanation is best? One criterion, with a long intel- 
lectual history dating back to William of Ockham and 
beyond, is that we should prefer explanations that cap- 
ture as much as possible with the fewest assumptions. 
Experimental work with adults and children indicates 
that people do indeed prefer explanatory simplicity 
(Lombrozo, 2016) but that they are also drawn to expla- 
nations that are as integrated as possible (Lombrozo & 
Vasilyeva, 2017). For example, people are more con- 
vinced by an explanation of weight loss and fatigue that 
does not explain each symptom independently (loss of 
appetite coincidentally combined with insomnia) but 
rather points to a single underlying cause (e.g., depres- 
sion). Explanations are also preferred to the degree that 
they cover a diverse range of phenomena (Kim & Keil, 
2003), and this is particularly true when people are 
prompted to pay attention to evidential diversity (Pres- 
ton & Epley, 2005). 

How are considerations such as simplicity and explan- 
atory breadth connected to Bayesian explanations of 
reasoning? One direct connection is that a preference 
for simple explanations (where simplicity is judged 
by the length of description in a coding language) is 
equivalent to choosing the most probable explanation 
(according to an appropriate prior) (e.g., Chater, 1996). 
Moreover, Bayesian philosophy of science has attempted 
to recast a variety of explanatory virtues as corollaries 
of a Bayesian analysis (e.g., Bovens & Hartmann, 2003; 
Howson & Urbach, 1989), although whether this project 
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is successful has been challenged (Douven & Schupbach, 
2015; but see Wojtowicz & DeDeo, 2020). 

So far, we have focused on cases where the premises 
from which reasoning proceeds are taken as given. But 
people are also active investigators into the world. This 
raises the question of how people should select infor- 
mation, or conduct experiments, that may be the basis 
for further reasoning. Initial work in the psychology of 
reasoning (e.g., Wason, 1968) presupposed that enquiry 
should seek only to falsify generalizations, on the princi- 
ple that only falsification can give certainty (in the spirit 
of Popper’s [1934/1959] falsificationist scientific method- 
ology). But from a Bayesian point of view, both discon- 
firming and confirming evidence can lead us to update 
our beliefs. Thus, under some circumstances, attempts to 
confirm the hypothesis may be particularly fruitful. 

Suppose we wish, for example, to investigate the 
somewhat fanciful hypothesis that eating blueberries 
turns your hair blue temporarily. According to the logic- 
based falsificationist search strategy, the sole objective is 
to find someone who has eaten blueberries and whose 
hair is nonblue and hence to search among blueberry 
eaters, on the one hand, and people with nonblue hair, 
on the other. Yet the latter search is likely to be futile— 
most people have nonblue hair and have also not 
recently eaten blueberries. On the other hand, it may 
well be worth asking the much smaller set of people with 
blue hair whether they have recently eaten blueberries; 
finding even a few of these might make one suspect that 
the rule has some validity. 

From a Bayesian point of view, the objective of inquiry 
will often simply be to maximize the amount of infor- 
mation gained from sampling new evidence; of course, 
before gathering evidence, we cannot know how infor- 
mative it will be. But Bayesian optimal data selection 
(Lindley, 1956) recommends that we maximize the 
expected amount of information gained (where the 
expectation is taken in the light of our probabilistic 
assumptions about the world). This is the starting point 
for a rational Bayesian explanation of what was pre- 
viously viewed as “confirmation bias” in data acquisi- 
tion tasks, including most notably Wason’s (1968) card 
selection task (Oaksford & Chater, 1994). According to 
the Bayesian framework, the optimal strategy should 
depend on probabilistic background assumptions (e.g., 
the probability of having blue hair, in the example 
above), and these do, indeed, seem to moderate perfor- 
mance in data selection tasks. 

From the standpoint of the logical paradigm, it is natu- 
ral to see a profound division between deductive reason- 
ing (guided by logical rules) and inductive reasoning 
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(where conclusions extend beyond the logical conse- 
quences of the premises). But, as we have seen, both types 
of process can be modeled in a uniform way in a Bayesian 
framework. Indeed, it seems natural to see both types of 
reasoning as inherently uncertain and dependent on rich 
background knowledge, rather than purely on the ver- 
bal premises that are given in the problem. The Bayesian 
approach, therefore, seems to sit uneasily with the pro- 
posal that the mind combines a slow, logical reasoning 
system with fast associative processes for inductive rea- 
soning (e.g., Heit & Rotello, 2010; Klauer, Beller, & Hiitter, 
2010; Rips, 2001; Stanovich, 2011). Interestingly, recent 
applications of signal detection theory to assess whether 
deductive and inductive reasoning are supported by sepa- 
rate processes or a single system appear to favor the latter 
(Stephens, Dunn, & Hayes, 2018). 


6. Social Reasoning: Argumentation 


The psychology of verbal reasoning has traditionally been 
pursued as if it were only concerned with the cognitive 
processes of isolated individuals. Yet verbal reasoning, like 
verbal behavior of all kinds, appears naturally to have a 
social role: as involved in communication, persuasion, 
and argumentation (Hahn & Oaksford, 2007; Mercier 
& Sperber, 2011, 2017). Indeed, understanding verbal 
reasoning in its communicative context may be crucial 
to understanding even the most elementary reasoning 
and to interpreting the significance of the statements 
over which reasoning occurs (Hilton, 1995). Moreover, 
the process of pragmatic enrichment may itself involve 
highly complex reasoning processes (Grice, 1975; Levin- 
son, 2000; Sperber & Wilson, 1986), which are often of far 
greater sophistication than the patterns of reasoning (MP, 
MT, etc.) that are ostensibly being probed in experimen- 
tal tasks. Recall that on hearing I didn’t get hot water this 
morning, we immediately infer that the speaker turned the 
hot tap in the right direction, ran it for sufficiently long, 
and then discovered the unexpected lack of hot water. We 
assume, too, that she obtained cold, or perhaps lukewarm, 
water from the tap, rather than no water all—otherwise, 
some stronger statement (e.g., There was no water at all 
this morning) would have been given. Speaker and hearer 
have to do significant inferential work to align on an 
interpretation—and this process can misfire. It could be, 
for example, that the speaker is referring to the water at 
the gym, while the listener assumes they are referring to 
the water at home. The wider point is that verbal reason- 
ing, as traditionally understood, presupposes that a great 
deal of “hidden” inferential work has already been carried 
out (Stenning & Cox, 2006). 


Bayesian Rationality in the Psychology of Reasoning 


Beyond questions of agreeing on the interpretation 
of what has been said, communication in general, and 
argumentation in particular, is invariably knowledge 
rich: whether new arguments or evidence will convince 
a person to change their beliefs can, in principle, depend 
on the entirety of their current knowledge. Argumenta- 
tion is also local and interactive: interlocutors make a 
series of specific argumentative moves, which may be 
either accepted or else repelled by further moves, which 
may themselves be countered, and so on. 

How far can a Bayesian approach extend from verbal 
reasoning to model argument more broadly? This has 
been a major focus of research for at least 15 years, focus- 
ing in particular on the question of whether so-called 
fallacies of informal argument, which are nonetheless 
widely deployed in real conversations, have a rational 
Bayesian foundation that may legitimately lead an audi- 
ence to change their degrees of belief (Hahn & Oaksford, 
2007; Oaksford & Hahn, 2004). 

The range of fallacies that have been considered is 
broad, including arguments from ignorance, ad homi- 
nem arguments, and many more. Here, we focus on key 
examples: slippery-slope and circular arguments. 

Consider two slippery-slope arguments (SSAs), both 
of which have been used in popular debate but differ- 
ing in strength—from the patently ridiculous (6) to the 
more credible (7): 


(6) If we allow gay marriage, then people will want to 
marry their pets. 

(7) If voluntary euthanasia is legalized, then involun- 
tary euthanasia will be. 


From a Bayesian perspective, SSAs can be viewed as 
founded on a cost-benefit analysis (Corner, Hahn, & 
Oaksford, 2011; Hahn & Oaksford, 2007): do not take the 
relevant action (p: allow gay marriage) unless its benefit 
(U(p)) exceeds the expected cost of the consequence (q: 
allowing interspecies marriage) to which it might conceiv- 
ably lead (Pr(q | p)U(q)). Thus, SSAs are a species of warning 
(Bonnefon, 2009). A key difference between (6) and (7) is 
that the probabilities of the consequences seem very dif- 
ferent: the chance of significantly raising the probability 
of interspecies marriage is infinitesimally small, whereas 
the chance of the extension of euthanasia to at least some 
nonvoluntary cases may be judged nontrivial (irrespective 
of how we evaluate either of these consequences). 
Interestingly, SSAs can depend crucially on the degree 
to which our categories are flexible (Hahn & Oaksford, 
2007). One reason that the jump from gay marriage to 
interspecies marriage seems so ludicrous is that these 
cases are judged to be highly dissimilar, and moreover, 
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the category of allowable marriages is itself quite rigid (it 
is not a matter of casual interpretation but is rooted in 
the law). On the other hand, different types of euthanasia 
may seem far more similar—and the category boundary 
(e.g., between quite what counts as voluntary or invol- 
untary) may seem anything but rigid. More broadly, the 
similarity between p and q affects people’s willingness 
both to assign them to the same category and to endorse 
the corresponding SSA (Corner et al., 2011; see also Rai & 
Holyoak, 2014, in the context of moral hypocrisy). 

The “fallacy” of circularity (Hahn, 2011) is often 
viewed as problematic even though it is logically valid. 
To take the extreme case, If p then p is not objectionable 
because it succumbs to logical counterexamples—indeed, 
it is tautologically true. Instead, circular reasoning can 
seem argumentatively useless—it presupposes precisely 
what is to be proved. But how the presupposition oper- 
ates can significantly affect how fallacious they appear 
to be, from the patently unconvincing (8) to the rather 
more compelling (9): 


(8) God exists because the Bible says so and the Bible is 
the word of God 


(9) Electrons exist because they make 3-cm tracks in a 
cloud chamber 


A good argument, of any kind, should have the potential 
to change our beliefs (Harman, 1986). (9) is a type of 
inference to the best explanation: if we suppose the exis- 
tence of electrons, that would explain (via a good deal of 
collateral knowledge about physical laws, the operation 
of cloud chambers, etc.) the 3-cm tracks; we can’t think 
of any other compelling explanation for the observa- 
tion of these tracks, so we infer that electrons do indeed 
exist. Thus, (9) concludes that electrons exist by assum- 
ing a theory that requires the existence of electrons and 
tracing its consequences, and so some nontrivial work 
has been done (although this work is “hidden” in the 
details of rival scientific theories and their predictions, 
which is not, of course, visible in the verbal statement 
of the argument). Indeed, (9) illustrates what appears 
to be a benign circularity, inevitable in abductive argu- 
ment, whether concerning science or not: to explain 
an observation, we postulate an explanation that is 
then bolstered (or disconfirmed) to the degree that the 
explanation captures the observation of interest. (8) is, 
by contrast, much less compelling: there is no bolster- 
ing of the explanation by the observation—in essence, 
because the prediction is not a distinctive prediction. So, 
while rival physical theories will struggle to explain the 
observation of cloud chamber tracks, it is all too easy for 
any number of nontheological theories to explain why a 
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religious text will claim divine authorship (these types of 
considerations can be captured in a hierarchical Bayes- 
ian model; Hahn, 2011; Hahn & Oaksford, 2007). Thus, 
the perceived strength of a circular argument depends 
not on its structure but on the credibility and number 
of possible alternative explanations (Hahn & Oaksford, 
2007, experiments 1 and 2). 

More broadly, from a Bayesian point of view, the 
study of verbal reasoning and the study of argumen- 
tation can be addressed using the same methods and 
are continuous, rather than corresponding to distinct 
domains. Indeed, despite the historical priority of the 
study of individual reasoning, it may be that commu- 
nication and argumentation are more fundamental and 
that individual reasoning can only be understood in a 
social, communicative context (Hahn & Oaksford, 2007; 
Hilton, 1995; Mercier & Sperber, 2011, 2017; Oaksford & 
Chater, 2020). 


7. Conclusion 


Bayesian theories are now widespread in the psychology 
of verbal reasoning and argument, from what have tradi- 
tionally been viewed as deductive reasoning tasks, to the 
study of induction, abduction, and argumentation. In 
each case, reasoning is presumed to be probabilistic and 
to depend not merely on the stated premises but typi- 
cally on the totality of a person’s beliefs. The purpose of 
reasoning of all kinds in daily life is, we have suggested, 
the updating of belief, rather than, for example, the 
evaluation of the validity of links between specific prem- 
ises and conclusions (although this may be crucial in 
important special cases, such as checking mathematical 
proofs, legal arguments, and the predictions of scientific 
theories). Traditional logic-based models of reasoning, 
by contrast, typically focus on validity and reasoning 
from the information given. The complexities of seman- 
tic and pragmatic interpretation required to even start 
such reasoning, the relevance of unstated background 
knowledge, and the social and argumentative context 
have traditionally been viewed as rather difficult-to- 
eliminate confounding factors. But from a Bayesian 
point of view, these are not confounds but the very topic 
of interest. Indeed, more broadly, Bayesian approaches 
to verbal reasoning hold the promise of integration 
with the wider program of modeling world knowledge, 
social interaction, communication, and the basic cogni- 
tive mechanisms of perception, categorization, memory, 
and motor control within a single theoretical framework 
(Chater & Oaksford, 2008; Tenenbaum, Kemp, Griffiths, 
& Goodman, 2011). 
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Notes 


1. As we have pointed out before (e.g., Oaksford & Chater, 2010), 
this starting point, especially for the conditional, contrasts starkly 
with the philosophy of logic and language (Nute, 1984), which 
abandoned the truth-functional view with the advent of the 
possible-worlds semantics for the conditional (Stalnaker, 1968). 


2. We refer the reader to Oaksford and Chater (2020) for a 
more extensive review of the issues raised here. 


3. An intuitive explanation of the origin of this interval is that 
Pr(q | not-p) can range between 0 and 1, the former yielding the 
lower bound and the latter the upper bound. 


4. Douven and Verbrugge (2013) argue that the triviality argu- 
ments rely on the assumption that the conditional-forming 
operator is independent of belief states (van Fraassen, 1976). 
They present evidence that this is not the case because people 
do not judge the conditional probability of a conditional, that 
is, Pr(if p then q|r), to be equal to the conditional probability 
Pr(q |p & 7). 
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4.6 Probabilities and Conditionals 


Klaus Oberauer and Danielle Pessach 


Summary 


What do conditionals have to do with probability? After a 
short introduction, the second section of this chapter will 
explain this connection and how it laid the groundwork 
for the “probabilistic turn” in the psychology of condi- 
tional reasoning. The rest of the chapter is devoted to two 
questions: (a) What is the meaning of conditionals, and 
(b) how do we reason with conditionals? We will answer 
these two questions from the perspective of probabilis- 
tic accounts of conditional reasoning. The third section 
is concerned with the probabilistic interpretation of the 
conditional, whereas the fourth section will cover current 
approaches to reasoning with (probability-)conditionals. 


1. Conditionals and Rationality 


Conditionals lie at the heart of human rationality. We 
use them to assert how states and events depend on each 
other—causally, logically, deontically, or in other ways 
(e.g., “If you want to learn more about conditionals, you 
should read this chapter”). We use them to express hypo- 
thetical thinking (e.g., “If we reduce CO2 emissions now, 
we can still keep global warming below 2°C”). They figure 
centrally in two major systems for formalizing rationality, 
formal logic and the probability calculus: first, “if-then”— 
defined as the material conditional—was included as one 
of the junctors linking elementary propositions in propo- 
sitional logic. Subsequently, the interpretation of “if-then” 
expressions as material conditionals has been criticized, 
leading to a “probabilistic turn” that links conditionals 
closely to conditional probabilities. This chapter reviews 
the arguments motivating this “probabilistic turn” and 
the related psychological research on how people interpret 
and reason with conditionals. 


2. Why Turn to Probabilities? 


The groundwork for the probabilistic turn was laid by the 
inaptitude of classical logic! to account for how people 


interpret conditionals, as well as for how they reason with 
them. Classical logic conceptualizes a conditional AC 
(read as: “If A then C,” where A is called the antecedent 
and C is called the consequent) as the material conditional. 
Whether a material conditional is true or false is defined 
by the truth-values of its constituents; it is therefore fully 
truth-functional. A truth function can be represented as a 
truth table, which maps the truth-value of a compound 
sentence such as “If A then C” as a function of the truth 
conditions of its components A and C. Table 4.6.1 shows 
the truth table for the material conditional. The material 
conditional is false when A is true and C is false and true in 
all other possible cases. This definition has consequences 
often characterized as paradoxical. From any true C, we 
can validly infer AC: from “Cats are mammals” we can 
infer “If cats are fish, then cats are mammals.” Another 
paradox: from =A (“not-A”; 7 is used to denote a nega- 
tion), we can validly infer AC. For example, “The moon 
is square” is clearly false; therefore, “If the moon is square, 
the president is a good person” and “If the moon is square, 
the president is a bad person” both follow. In the material- 
conditional account, A—>C and A~~7-C can both be true. 
Psychological research has shown that people do not 
understand conditionals as material conditionals. One 
possibility to assess how people understand AC is to 
ask them to indicate for each possible combination of 


Table 4.6.1 
Truth table for the material conditional and responses in 
Johnson-Laird and Tagart (1969) 


Conditional (A = C) 


A Cc MC Responses 
True True True True 
True False False False 
False True True Irrelevant 
False False True Irrelevant 


Note: MC = material conditional. Responses are participants’ 
majority responses. 
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truth-values for A and C (i.e., for each line of the truth 
table) whether this combination renders the conditional 
true or false or is irrelevant to the truth of the conditional. 
This task is referred to as the truth table task. The usual pat- 
tern is displayed in table 4.6.1: participants tend to judge 
the false-antecedent cases as “irrelevant” rather than as 
cases rendering the conditional true (Evans, Handley, 
Neilens, & Over, 2008; Johnson-Laird & Tagart, 1969; 
Schroyens, 2010). 

People also do not reason with conditionals as dic- 
tated by classical logic. In the psychology of reasoning, 
four simple inference forms—taking a conditional major 
premise and a categorical minor premise—are frequently 
studied. They are termed “modus ponens” (MP), “modus 
tollens” (MT), “affirmation of the consequent” (AC), and 
“denial of the antecedent” (DA). For example, MP in its 
abstract form is: from “If A then C” and A follows C. The 
full list of the four inferences is as follows: 


MP: A>C AC: A>C 
A C 
ca A 

MT: A>C DA: A>C 
7C 7A 

7A 7C 


Note: Statements above the horizontal lines are premises, below 
the line are conclusions. 


While MP and MT are valid according to classical logic, 
AC and DA are not. However, in a conditional inference 
task (i.e., a task where participants have to judge the four 
inference forms as valid or not), people usually do not 
adhere to that norm: although almost everyone endorses 
MP as valid (~ 95%), the endorsement rate drops signifi- 
cantly for MT (~ 70%), and the so-called fallacies (AC 
and DA) are accepted as valid with rates well over 50% 
(e.g., Evans, Newstead, & Byrne, 1993; Evans, Over, & 
Manktelow, 1993; Schroyens & Schaeken, 2003). 

The paradoxes of the material implication, its incom- 
patibility with people’s judgment (in the truth table task 
as well as in the conditional inference task), and the fact 
that everyday reasoning seldom deals with “true” and 
“false,” but rather with uncertainty in different degrees, 
have led philosophers as well as psychologists to ques- 
tion the material conditional as an explication of the 
meaning of “if-then” statements. As an alternative, the 
“probability conditional” has been suggested (Adams, 
1975; Bennett, 2003; Edgington, 1995). 
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3. Probabilistic Account for the Meaning 
of Conditionals: The Probability Conditional 


The probability conditional is defined by “the Equation”: 
P(A C) = P(C\A). 


The Equation defines the probability of the conditional 
as the conditional probability of the consequent given 
the antecedent. It was inspired by an account of how 
people assess conditionals, known as the Ramsey test. 
In probably one of the most famous footnotes in the 
history of philosophy, Ramsey wrote, “If two people are 
arguing ‘If p will q?’ and are both in doubt as to p, they 
are adding p hypothetically to their stock of knowledge 
and arguing on that basis about q.... We can say they 
are fixing their degrees of belief in gq given p” (Ramsey, 
1929/1990, p. 247). So, when we assess AC, we sup- 
pose A by adding it (hypothetically) to our beliefs about 
the world and assess, from that hypothetical belief state, 
how likely C is. The outcome of the Ramsey test is the 
conditional probability of C, given A, P(C|A). 

Before we continue to assess the merits and draw- 
backs of the probability conditional, some clarifications 
and specifications are necessary. First, when we speak 
of probabilities, we mean subjective probabilities, some- 
times also called degrees of belief. 

Second, while probabilities of conditionals equal con- 
ditional probabilities, they do not inherit all their (math- 
ematical) features. The following two points should 
make this distinction clear: (1) conditional probabili- 
ties are mathematically defined by the Ratio formula: 
P(C|A) = P(A&C)/P(A). This does not apply to condi- 
tionals: for the purpose of determining P(A > C), P(C|A) 
is defined as the outcome of the Ramsey test, that is, it is 
the result of assessing the probability of C on the supposi- 
tion of A. Edgington (1995) gives the following example: 
we can be thinking about what to do next, A or B, consid- 
ering what consequences those options would have and 
how likely they are. Obviously, we can do that without 
having fixed our degree of belief in A—after all, we are 
still deliberating whether to make A true. Hence, we 
can determine P(C|A) without having a belief about P(A) 
(Edgington, 1995, p. 266). 

(2) Another consequence of the Ratio formula is that 
P(C|A) is not defined when P(A) = 0. Stalnaker (1968) 
offers an extension of the Ramsey test that deals with 
cases where P(A) = 0 (i.e., counterfactual conditionals). If 
A is incompatible with our knowledge, we cannot sim- 
ply add it to our stock of knowledge and still be coher- 
ent in what we believe. Stalnaker’s extended Ramsey test 
requires that, before adding A, we make the minimally 


Probabilities and Conditionals 


necessary adjustments to our knowledge to make it com- 
patible with A. However, this extended Ramsey test has 
been criticized, mostly because it is impossible to deter- 
mine which adjustments to one’s stock of knowledge 
are “minimal.” Bennett (2003), among others, therefore 
maintains that when P(A) = 0, conditionals—with a few 
exceptions—are not defined and have no probability. 

Now that the Equation is disambiguated, we can 
turn to its main implication: conditionals have no truth 
conditions. 


3.1 A Triviality Result 

The triviality result is Lewis’s (1976) proof that there is no 
combination of cases in the truth table for which the joint 
probability systematically equals P(C|A). Because any 
proposition conjoining A and C is defined by its truth- 
value, which in turn is a function of the truth-values of A 
and C, it follows that there is no proposition that satisfies 
the Equation (Lewis, 1976). Hence, if the meaning of a 
conditional such as “If A then C” is defined by the Equa- 
tion, it cannot be a proposition (Adams, 1975; Bennett, 
2003; Douven & Verbrugge, 2010; Edgington, 1995; Lewis, 
1976). The consequences are far-reaching: if we accept the 
Equation, conditionals do not have truth conditions. This 
seems to be a high price to pay for an explanation of the 
conditional. For instance, it would not be meaningful to 
say that a conditional is true or false, and we could no lon- 
ger characterize mental states such as believing or doubt- 
ing a conditional as propositional attitudes (i.e., attitudes 
about a proposition). Why should we pay that price? 


3.2 Arguments for the Probability Conditional 

Let us first see how the probability conditional fares 
with regard to the objections against the material con- 
ditional. We saw above that the material conditional has 
some counterintuitive implications when the antecedent 
is false or when the consequent is true. Considering an 
antecedent that is certainly false (i.e., P(A) = 0), we can 
follow Bennett and regard the conditional as undefined 
or follow Stalnaker’s extended Ramsey test and regard the 
conditional as more or less credible depending on how 
high P(C|A) is after we have adjusted our beliefs to accom- 
modate A. Either version saves us from having to endorse 
a conditional simply because its antecedent is false. Con- 
sidering a case where the antecedent A is just very unlikely, 
but P(A) >0, the Equation ensures that nothing paradoxi- 
cal follows: P(C|A) can still take on any value. 

The same rationale applies to “true” consequents, 
when they actually are just very likely. It might well be 
that Peter always goes for a run before he goes to work, 
and those who know him think it very likely that he goes 
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for a run. However, “If Peter is injured, he goes for a run 
before he goes to work” is utterly unlikely. This example 
demonstrates that a high P(C) does not imply a high 
P(C|A), so that the probability conditional avoids the 
first paradox of the material conditional in most circum- 
stances. Only when the consequent is a necessary truth, 
such that P(C) = 1, we are also forced to accept the con- 
ditional, because the conditional probability must be 1.” 

Here comes another paradox of the material condi- 
tional: a material conditional “If A then C” is false if and 
only if A and -C, so from the falsity of “If A then C,” 
we can infer that A and ~C. Most people would agree 
that “If Peter is in London, then he is in France” is clearly 
false, regardless of whether Peter is in London or not. Yet 
the material-conditional account would require Peter to 
be in London for the conditional to be false. The prob- 
ability conditional does not have that problem. In order 
to understand that, we have to clarify a consequence 
of the fact that conditionals have no truth conditions: 
when we ask, “What is the probability of AC?” we do 
not ask, “What is the probability that it is true?” A prob- 
ability conditional can, of course, be true (when anteced- 
ent and consequent have obtained), and it can be false 
(when the antecedent has obtained but the consequent 
has not). Therefore, it does have a truth-value when the 
antecedent holds. However, in case the antecedent did 
not obtain (yet), it is neither true nor false; it is simply 
more or less likely. So one can restate the question how 
probable we think AC is as how probable we think it 
is true given that it has a truth-value. The conditional has 
a truth-value when A is true, and in that case, it is true 
when A & C is true. Hence, the probability of AC, given 
that it has a truth-value, is P(A&C)/P(A), and that, of 
course, matches the conditional probability (Edgington, 
1995). Turning back to our London-France example, we 
see that we have an extreme case: P(C | A) = 0. In this case, 
we can say that the conditional is “false” to express that 
our degree of belief is very low,’ but in doing so, we do 
not assign the conditional a truth-value but refer to the 
conditional probability P(C|A) instead (Jeffrey, 1991). We 
use “false” in a pleonastic sense, not referring to the value 
“false” in the truth table but to the zero probability of the 
conditional (Over & Baratgin, 2017). There are additional 
philosophical arguments in favor of the probability con- 
ditional (for a more detailed account, see Bennett, 2003). 
We now turn to the experimental evidence. 


3.3. Empirical Evidence for the Probability Conditional 

As mentioned above, participants in psychological exper- 
iments do not judge that false-antecedent cases render 
the conditional true but judge those cases to be irrelevant 
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for the truth of the conditional (Johnson-Laird & Tagart, 
1969). This is in line with the probabilistic account, 
because the fact that A is false has no bearing on P(C|A). 
However, this finding can also be explained by an alterna- 
tive theory that builds on the material conditional for its 
assumptions on how people represent conditional state- 
ments: mental model theory (Johnson-Laird & Byrne, 
2002).* This theory is currently the main competitor to 
the probabilistic approach in the psychology of reason- 
ing, and therefore we briefly introduce it next. 

Mental model theory (MMT) assumes that the mate- 
rial conditional expresses the fully elaborated core 
meaning of conditionals. In addition, the theory speci- 
fies how conditionals are mentally represented and the 
psychological processes operating on them when people 
make judgments about conditionals or engage in con- 
ditional reasoning. MMT states that conditionals are 
represented as a set of mental models. A complete repre- 
sentation of a “basic” conditional (i.e., a conditional for 
which no contextual or knowledge factors modulate its 
core meaning) includes one model for each truth table 
case that makes the material conditional true (i.e., all 
cases that are possible, assuming the conditional is true): 


A C 
7A C 
aA OAC 


In this notation, each line is a mental model represent- 
ing one possible state of the world. Note that only “true” 
rows of the truth table are represented as models, not 
the false ones (i.e., “A&7C” is not represented because 
it would render the conditional false; Johnson-Laird & 
Byrne, 2002). MMT also states that usually people only 
represent the first model explicitly and add the others 
implicitly, leaving them unspecified: 


A © 
[.. J 


The three dots in square brackets are an “elliptical” 
representation of the unspecified cases. The full set of 
three models can be fleshed out from the initial repre- 
sentation, but that requires additional mental capacity 
and therefore is often not done. 

These assumptions provide an explanation of why 
people tend to judge false-antecedent cases as “irrel- 
evant” for the truth-value of conditionals in truth table 
cases: people fail to flesh out the full set of models 
and therefore cannot fit the ~A-cases into any mental 
model of the conditional (Johnson-Laird & Byrne, 
2002). However, this explanation has a problem: MMT 
also predicts—correctly—that people judge the con- 
ditional as “false” in case of A&-C. Yet, this “false” 
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case of the truth table is not explicitly represented as 
a mental model, and as such, it has the same status as 
the —A-cases with respect to the initial, sparse represen- 
tation in which only the A&C-case is explicitly repre- 
sented. Inferring that A&-C is incompatible with the 
conditional requires fleshing out the fully explicit set of 
three models (Johnson-Laird & Byrne, 2002). There is no 
mechanism that can explain how, when only the initial 
“A&C” model is explicitly represented, A&-C can be 
distinguished from ~A&C and ~A&~-C. Therefore, in 
the truth table task, participants should be predicted to 
judge all three cases as irrelevant or all three as false. 

The classic truth table task provides only indirect evi- 
dence for the probabilistic account of conditionals. A 
more direct test of the Equation would be to see whether 
participants really adhere to it in their judgments of 
probabilities. 

Evans, Handley, and Over (2003) and Oberauer and 
Wilhelm (2003) examined whether participants’ judg- 
ments about the probability of a conditional “If A then 
C” corresponded to the conditional probability P(C | A) or 
to the probability according to the material-conditional 
account (i.e., 1-P(A&-C)). Participants were tested on 
abstract conditionals concerning a set of cards that had, 
for example, colored letters printed on it (e.g., “If there is 
an A on the card, it is blue”). For each conditional, they 
were given the relevant frequencies of the four truth table 
cases (e.g., of the 100 cards, 50 had a blue A, 10 a red A, 
20 a blue B, and 20 a red B printed on it). Participants 
were then asked to estimate the probability of the condi- 
tional. Both studies found strong evidence for the proba- 
bilistic account and against the material conditional: the 
probability of the conditional was highly correlated with 
P(C|A), and a majority of participants was classified as 
adhering to the Equation. 

Over, Hadjichristidis, Evans, Handley, and Sloman 
(2007) went a step further: instead of giving the relevant 
frequencies of the truth table cases, they used everyday 
conditionals and had participants judge the respective 
probabilities (with the restriction that they had to sum 
to 100%). Only the conditional probability (computed 
from the probability ratings for individual events) was 
correlated with the rated probability of the conditional. 
However, Evans and colleagues (2003; Evans, Handley, 
Neilens, & Over, 2007) as well as Oberauer and Wilhelm 
(2003) also found evidence of a conjunctive interpreta- 
tion of conditionals: for a substantial subgroup of par- 
ticipants, P(A—C) was mostly influenced by P(A &C). 
This pattern of probability judgments is referred to as 
the “conjunctive pattern.” 

How can we explain the “conjunctive pattern” of 
probability judgments? Clearly, people do not think 
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A&C to be equal to AC, as a person equating the two 
would be utterly incapable of hypothetical thought.° It 
could be argued that the effect is task specific: in the 
tasks in which this pattern occurred, participants always 
had to judge how likely a conditional was true for a ran- 
domly drawn subset of cards. If there are participants 
who regard the conditional as true only if at least one 
of the cards was an “A &C” card, this could explain the 
conjunctive pattern. Another possibility, suggested by 
Evans and colleagues (2003), is that this pattern reflects 
an incomplete Ramsey test. Participants set out to assess 
the probability of C on the supposition of A (i.e., once 
A is added to the stock of knowledge). Doing so requires 
one to compare the probability of resulting C-cases (i.e., 
A&C) with that of the resulting ~C-cases (i.e., A&C). 
If participants stopped halfway during this procedure, 
omitting the computation of P(A&-C), that could 
explain why they come up with P(A & C) as the probabil- 
ity of AC. Oberauer, Geiger, Fischer, and Weidenfeld 
(2007) tested these two explanations but rejected both. 
There are two other explanations for the conjunc- 
tive pattern: Edgington (2003) suggests that the word- 
ing of the question is ambiguous. As made clear above, 
the probability of a conditional is not the probability 
that it is true. Yet this is exactly what was asked in the 
experiments. So some participants could have ignored 
the “true” at the end of the sentence and answered 
with P(C|A), while others took the question literally 
and hence responded with the only cases where the 
conditional is true, that is, the “A &C’-cases. As far as 
we know, this hypothesis has not been tested directly; 
however, Oberauer and Wilhelm (2003) found no con- 
junctive pattern when participants were not required 
to answer with a probability estimate but simply had to 
decide whether a conditional was true or false. According 
to Edgington’s explanation, this question should have 
elicited predominantly conjunctive responses, contrary 
to what was found. This leads us to the last attempt to 
explain the conjunctive pattern: the effect could be due 
to difficult arithmetic operations that are necessary to 
compute the conditional probabilities. There is some evi- 
dence favoring this explanation. As mentioned before, 
the conjunctive pattern disappeared when participants 
did not need to compute any probability but simply had 
to assess if their probability was high enough for them to 
accept the conditional as “true” (Oberauer & Wilhelm, 
2003). Similarly, Over and colleagues (2007) did not find 
evidence of a conjunctive interpretation with their every- 
day conditionals. Although their participants had to give 
a numerical estimation of the probability of the condi- 
tional, they did not have to calculate them on arbitrary 
numbers provided by the experimenters but could use 
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their own, not necessarily explicit, estimation. And last 
but not least, evidence supporting the arithmetic-load 
explanation comes from Fugard, Pfeifer, Mayerhofer, and 
Kleiter’s study (2011). They used a facilitated-response 
paradigm: participants were asked about probabilities 
of a six-faced die landing a certain way in an “x out of 
y” format (e.g., “2 out of 6”), where participants could 
just reuse the frequencies given. For example, frequen- 
cies could be “Two sides have a yellow circle, two a yel- 
low square, and two a red circle.” The estimate of the 
probability of the conditional “If the die shows a yellow 
side, it shows a circle” only requires minimal arithmetic 
ability (namely, adding the two yellow cases). Fugard and 
colleagues found that (a) overall, approximately 73% of 
the participants judged the conditional according to the 
conditional probability, and (b) this percentage increased 
over the 70 trials each participant had to go through: 
55% of the participants shifted to eventually adopt the 
conditional probability, most of which had adopted a 
conjunctive interpretation before. 

Overall, the experimental evidence for the interpreta- 
tion of our everyday conditional as the probability con- 
ditional is substantial (for further evidence stemming 
from negated conditionals, see, e.g., Handley, Evans, & 
Thompson, 2006). Recently, however, the probability 
conditional has been criticized for ignoring an impor- 
tant aspect of the meaning of conditionals: relevance. 


3.4 
“Tf Peter is wearing a blue shirt today, the sea levels will 
rise.” This conditional strikes most people as odd at least, 
because the antecedent has no apparent relevance for the 
consequent. Yet, under the material-conditional and 
probabilistic account, we are forced to accept it respec- 
tively as true or highly probable, simply because sea level 
rise is highly probable. What is wrong with this condi- 
tional is the missing semantic link between the anteced- 
ent and the consequent (and we therefore refer to it as a 
“missing-link conditional,” in accordance with Douven, 
2015). This is the main point of the critique of inferential- 
ism (Douven, 2015; Krzyzanowska, 2015). Inferentialism 
states that a conditional is only true if there is a suffi- 
ciently strong argument from its antecedent, plus rele- 
vant background knowledge, to its consequent (Douven, 
2015; Krzyzanowska, Wenmackers, & Douven, 2014). 
It requires an inferential connection between antecedent 
and consequent, which can be a deductive, abductive, or 
inductive one (Krzyzanowska et al., 2014). 
Skovgaard-Olsen, Singmann, and Klauer (2016) tested 
this requirement experimentally. They found that par- 
ticipants’ judgments were predicted almost perfectly by 
the Equation for regular conditionals, but this relation 
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failed for missing-link conditionals. To be precise, the 
relation failed for some participants (who, irrespective of 
the conditional probability, assigned low probability to 
missing-link conditionals), while others still adhered to 
the conditional-probability account. This group pattern 
was confirmed by Skovgaard-Olsen, Kellen, Hahn, and 
Klauer (2017), who found that only a minority (17%) 
of participants adhered to the Equation for missing-link 
conditionals. Another approach to empirically test infer- 
entialism against the probability conditional is offered 
by “and-to-if” inferences (also referred to as “center- 
ing”: from “A & C” follows “If A then C”). This inference 
is valid according to both material- and probability- 
conditional accounts but invalid for an inferentialist. 
Although people indeed endorse centering (Cruz, Barat- 
gin, Oaksford, & Over, 2015), that seems to be the case 
only with conditionals where the antecedent is relevant 
for the consequent and not with missing-link condition- 
als (Skovgaard-Olsen, Kellen, et al., 2017; Skovgaard- 
Olsen, Singmann, & Klauer, 2017). The main point of 
disagreement between proponents of the probability 
conditional and proponents of inferentialism is whether 
or not the effect of relevance is of a pragmatic or a seman- 
tic nature (i.e., whether or not it affects the meaning of 
the conditional), a question that is still hotly debated 
(see Douven, Elqayam, Singmann, & van Wijnbergen- 
Huitink, 2018; Krzyzanowska, Collins, & Hahn, 2017). 


4. Probabilistic Accounts of Reasoning 
with Conditionals 


As mentioned in the introductory section, if one accepts 
the material conditional as the meaning of conditionals, 
the usual pattern of people’s evaluations of conditional 
inferences implies that people are extraordinarily bad at 
reasoning: the only inference they are really good at is 
modus ponens. Mental model theory (Johnson-Laird & 
Byrne, 2002), which accepts the material conditional as 
the normative standard, explains these findings by the 
failure of people to fully flesh out the set of mental mod- 
els. MP and AC are endorsed if only the initial model 
(A &C) is represented. If participants manage to add a 
further model “-~A &C,” they correctly endorse MP and 
reject AC and DA. To correctly endorse MT, they would 
have to further add the model “=A &-=C,” because it is 
the only model of the conditional that is also compatible 
with the minor premise of the MT argument, “not C.” 
Hence, if the fully explicit set of three models is fleshed 
out, participants accept the valid inferences (MP and 
MT) and reject the fallacies (AC and DA), which would 
be the normatively correct response by the material- 
conditional interpretation. 
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As we have seen in the previous section, there is strong 
evidence that people interpret conditional statements 
according to the probability conditional. How do probabi- 
listic theories account for people’s pattern of endorsement 
and rejection of the four simple conditional inferences? 
One approach is to understand inferences as Bayesian 
updating of our knowledge in light of the premises (Oaks- 
ford & Chater, 2007). This “purely Bayesian” approach 
interprets the four inferences as inferring the conditional 
probabilities of the conclusion, given the minor (categori- 
cal) premise. For instance, confronted with a DA argument 
of the form “If A then C; not C; therefore not A,” reasoners 
are assumed to draw on their knowledge about C and A to 
assess the probability of “not A,” given “not C.” Oaksford, 
Chater, and Larkin (2000; see also Oaksford & Chater, 
2001) were the first to propose this interpretation as a for- 
mal reasoning model. To model the person’s knowledge, 
they require three parameters to capture the joint subjec- 
tive probability distribution of A and C (i.e., the four truth 
table cases), from which the respective conditional prob- 
abilities can be computed: e = 1- P(C| A), referred to as the 
“exceptions parameter,” a = P(A) and b = P(C). From this 
joint probability distribution, MP endorsement rate is 
predicted by P(C|A), MT by P(-A|=C), AC by P(A|C), and 
DA by P(-C| =A). 

Oberauer (2006) compared the Oaksford—Chater 
model with a formalization of MMT and found that 
the probabilistic account did not fare well in explaining 
people’s pattern of endorsement and rejection of condi- 
tional inferences. The problem for the Oaksford—Chater 
model is that to explain the high MP acceptance rate, it 
must estimate the exceptions parameter e to a very small 
value. This implies that the model also predicts high 
MT acceptance, contrary to the data. As a compromise, 
the model, when fit to data, has been found to under- 
estimate endorsement of MP and overestimate endorse- 
ment of MT inferences (Oberauer, 2006; Schroyens & 
Schaeken, 2003). Oaksford and Chater (2007, 2013) pre- 
sented a revision of their model in which they introduce 
a new exceptions parameter e’, derived by updating e on 
a single occurrence of a counterexample to the condi- 
tional (i.e., an occurrence of an “A and ~C” example). 
Whereas MP endorsement is still estimated by the “old” 
exceptions parameter e, estimation for all other infer- 
ences is based on the new parameter. This solves the MP- 
MT asymmetry problem, because MP and MT are now 
decoupled. 

Oaksford and Chater (2007, 2013) justify the intro- 
duction of e’ by arguing that the minor premises given 
in the context of inferences MT, AC, and DA provide 
information that is equivalent to presenting a counter- 
example to the conditional. Therefore, the endorsement 
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rates of these inferences should be predicted with a dif- 
ferent exceptions parameter. In our view, this justifica- 
tion is contrived: learning that “not A” or that “not C” 
does not provide reasons to question “If A then C.” No 
amount of people who are not hit by a car, or of people 
who are not injured, should undermine our belief in 
the conditional “If a person is hit by a car, they will 
be injured.” Therefore, although the revised model of 
Oaksford and Chater is better able to accommodate the 
data, it does so at the price of being less principled than 
the original model. 

Another purely Bayesian approach, similar to the 
original Oaksford—Chater model, is based on Bayesian 
networks, a probabilistic graphical model that repre- 
sents the dependencies between variables as a directed 
acyclic graph (Pearl, 2009). While the two approaches 
are similar, they are certainly not equivalent, but due to 
space restrictions, we cannot go into more detail here 
(for a detailed account, see chapter 4.2 by Hartmann and 
chapter 7.1 by Pearl in this handbook). One thing both 
approaches have in common is that they explain condi- 
tional reasoning as an inference from the minor premise 
to the conclusion. 

But then, what is the role of the conditional (i.e., 
the major premise) in the inference task? Or put differ- 
ently, what is the difference between inferring C from 
the two premises “If A then C” and A (i.e., the full MP 
argument) and inferring C from A alone (i.e., a reduced 
MP argument), as both are interpreted as estimating 
P(C|A)? Within the Bayesian framework, it is not clear 
how adding the conditional premise could play a role 
in knowledge updating. Normally, in Bayesian updat- 
ing, additional premises are conditionalized on, that is, 
we assess P(conclusion | all premises). However, condi- 
tionalizing is not possible when the additional premise 
is itself a conditional probability. Oaksford and Chater 
(2007) propose that the effect of adding a conditional 
premise should be reflected in a decreased exceptions 
parameter e = 1-P(C|A), that is, the degree of belief in 
the sufficiency of the antecedent for the consequent 
increases. Hartmann and Rafiee Rad (2012) propose that 
instead of only increasing P(C|A), all parameters captur- 
ing the joint probability distribution are adjusted so that 
the Kullback-Leibler distance between the old and the 
new distributions is minimized. 

Klauer, Beller, and Htitter (2010) investigated the effect 
of the conditional on the inferences by presenting to 
participants both reduced inferences (i.e., without the 
major premise) and full inferences (i.e., the standard MP, 
MT, AC, and DA). The reduced inferences are viewed as 
an indication of how much the conclusion is endorsed 
based only on background knowledge. For example, when 
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given the information that the streets are wet, most 
people would conclude that it probably has rained, even 
if they haven’t been told “If it rains, the streets are wet.” 
To model the effect of adding the conditional to the infer- 
ence, Klauer and colleagues (2010) developed the dual- 
source model (DSM), a mixture model that implements 
Oaksford, Chater, and Larkin’s (2000) probabilistic model 
to account for the reduced inferences (the “knowledge” 
component in the DSM terminology) but adds two com- 
ponents for the full inferences: (a) four form parameters, 
which capture to what extent the inference forms (i.e., 
MP, MT, AC, and DA) are seen as logically valid, and 
(b) a weighting parameter, which indicates how much 
weight is given to the form-based information relative 
to the knowledge-based information. Singmann, Klauer, 
and Beller (2016) compared the original and the revised 
Oaksford—Chater model (2000, 2007, 2013), the Bayes 
nets approach with Kullback—Leibler distance minimiza- 
tion (Hartmann & Rafiee Rad, 2012), and the DSM. They 
found that the DSM explains substantially more variance 
than the competing models and also confirmed a unique 
prediction of the DSM. Singmann and colleagues con- 
clude that the effect of adding a conditional goes beyond 
updating the knowledge base (i.e., the joint probability 
distribution). Instead, they argue, adding the conditional 
to the minor premises provides participants with an infer- 
ence form that is then evaluated in terms of logical sound- 
ness. It is important to note that logical soundness does 
not demand one to refer to bivalued logic: MP and MT are 
also probabilistically valid, whereas DA and AC are not (for 
an introduction to probability logic, see chapter 4.4 by 
Pfeifer, this handbook). 

Further experiments provide support for the DSM by 
demonstrating that the form parameters can be selec- 
tively influenced by experimental variations of the infer- 
ence form (e.g., to biconditional, “only if” vs. “if-then,” 
or adding premises). Conversely, the form parameters do 
not change when the inference form is held constant 
(Klauer et al., 2010; Singmann et al., 2016). One limi- 
tation of the DSM is that, as a measurement model, it 
serves well to measure the effect of inference form but 
does not provide an explanation of how the validity of 
the inference forms is assessed. 

Insight into what causes the effects of inference form 
could be gained by looking at a different class of theo- 
ries, known as “dual-process” theories of reasoning. Vari- 
ous dual-process theories exist, even in the small world of 
conditional reasoning, but we will focus only on the gen- 
eral idea (for a detailed account, see chapter 2.5 by Klauer, 
this handbook). As the name suggests, they assume two 
processes, one (usually called Type 1) fast, automatic, 
and intuitive, the other (usually called Type 2) slower, 
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controlled, and relying strongly on working-memory 
capacity. The dual-process framework thus naturally 
explains reasoning from the inference form in terms 
of the Type 2 process, whereas the knowledge compo- 
nent can be regarded as a Type 1 process. A candidate 
for a Type 2 process would be, for example, mental 
model theory, and indeed, Verschueren, Schaeken, and 
d’Ydewalle (2005) have proposed a dual-process model 
that integrates probabilistic inference as a Type 1 process 
and mental models as a Type 2 process. This model fared 
very well in a formal comparison of several competing 
models of conditional inferences (Oberauer, 2006). 

But, as we have seen above, mental model theory 
builds on the material conditional, an assumption that 
is not supported by the empirical data, so we are left 
with a model that accounts for conditional inferences 
but is based on an interpretation of the conditional 
that is untenable. Geiger and Oberauer (2010) have 
sketched one potential way to keep the explanatory 
power of MMT for conditional inferences while avoid- 
ing the material-conditional interpretation of “if-then” 
premises. According to this proposal, the meaning of a 
conditional is not directly captured by a set of mental 
models that represent its truth conditions—rather, it is 
incorporated in procedural knowledge. A conditional 
statement “If A then C” is understood as an instruc- 
tion to add an element C to every mental model of 
A in working memory. Conditional reasoning starts 
with building a mental model of the minor (categori- 
cal) premise. In case that model includes A, the pro- 
cedure implementing the conditional adds C to it. For 
instance, when thinking through an MP argument, the 
reasoner first builds a mental model of A in working 
memory, representing the minor premise. The condi- 
tional premise instructs a procedure to augment this 
model, resulting in the model “A &C,” from which the 
conclusion “C” follows immediately. Thinking through 
an MT argument involves more steps: to represent the 
minor premise, the first model is “~C.” The reasoner 
now tentatively augments this model by adding “A” 
and by adding “~A,” effectively reasoning from the 
supposition that A, or the supposition that not A. The 
first augmentation results in a model “A &-C,” which 
triggers the procedure that incorporates the conditional 
premise. That procedure adds “C” to the model, in con- 
flict with the “-C” element, and that conflict signals 
the contradiction between the supposition of A and the 
premises, which leads to elimination of the “A &~C” 
model. The remaining “~A&~C” model licenses the 
conclusion “not A.” 
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5. Conclusion 


Just rethink for a moment: if you had not known, would 
you have guessed that the probability of the conditional, 
judged by laypeople, is approximately equal to the condi- 
tional probability? On the one hand, this seems implau- 
sible, as calculating conditional probabilities is difficult. 
On the other hand, uncertainty pervades our knowledge 
of the world, and we have a rough sense of how likely 
certain events are to happen and how credible certain 
claims are. Probabilities are merely a way of expressing 
our degrees of certainty numerically. Moreover, a hall- 
mark of human rationality is the consideration of hypo- 
thetical scenarios (Evans & Over, 2004): when we plan 
a course of action, we think about what is likely to hap- 
pen if we do this or that. When we reflect on something 
that went wrong, we consider whether things would have 
turned out better if one or another unfortunate event had 
not happened. 

Thinking about hypothetical scenarios does not 
require the ability to calculate conditional probabilities. 
It requires to form a representation of the hypotheti- 
cal state—the supposition—and to use relevant knowl- 
edge to infer what is likely to happen, perhaps through 
a mental simulation (Hegarty, 2004). When people are 
asked how certain they are of their beliefs about what 
would happen, they can express their degree of certainty 
on a coarse scale of subjective probabilities; Khemlani, 
Lotstein, and Johnson-Laird (2015) have proposed a 
simple algorithm for arriving at such coarse probability 
estimates on the basis of mental models. Thinking about 
what happens, or would have happened, in hypotheti- 
cal scenarios is an assessment of conditional probabili- 
ties: how probable is outcome C, given the hypothetical 
event or action A? Speaking about such hypothetical 
scenarios involves using conditionals (e.g., “If I go home 
early, my boss won't be pleased. I better stay another 
hour”). Therefore, a close link between subjective con- 
ditional probabilities and the meaning of a conditional 
statement appears plausible. 

The probability conditional and the Equation have 
received compelling support from studying how people 
understand conditionals (Handley et al., 2006; Oberauer 
& Wilhelm, 2003; Over et al., 2007). At the same time, 
the question of how people reason with conditionals is 
still unresolved, both on the normative and the descrip- 
tive level. On the normative side, the purely Bayesian 
approaches have no rational procedure for updating 
knowledge, given conditional premises. Probability logic 
could be a possible answer to that problem, but that 
would mean abandoning the purely Bayesian approach 
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to reasoning. On the descriptive side, people’s rates 
of endorsement of conditional inferences under vari- 
ous conditions are explained only poorly by Bayesian 
updating models. Dual-source or dual-process models, 
in which Bayesian updating represents the knowledge- 
based process, appear more promising. These models 
require more work devoted to spelling out the form- 
based process in which reasoners use conditional prem- 
ises to inform their inferences. 


Notes 


1. By “classical logic,” we are referring to the standard bivalued 
logic, like, for example, first-order or propositional logic. 


2. This has been the source of some criticism of the probability 
conditional, as we will see later on. 


3. Obviously, the same rationale applies to “true” when our 
degree of belief is very high. 


4. There have been various attempts at a substantial revision of 
mental model theory; see, for example, Barrouillet, Gauffroy, 
and Lecas (2008) and Johnson-Laird, Khemlani, and Goodwin 
(2015). For a discussion of the plausibility of these revisions, 
see Oberauer and Oaksford (2008) and Baratgin et al. (2015), 
respectively. 


5. This person would, after hearing the warning “If you touch 
the wire, you will get an electric shock,” answer “No, that is not 
true” because he does not touch the wire and is not shocked. 


6. We do not suggest that people ask themselves first “What 
is the probability of A&C?” and “What is the probability of 
A&7C?” and then compute a conditional probability (see 
our remarks on the Ratio formula above) but that people can 
directly assess P(C|A) if they have the relevant background 
knowledge, as is the case in Over and colleagues’ experiment, 
which used everyday causal conditionals. 
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4.7 Representing Belief: Beyond Probability and Logic 
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Summary 


This chapter surveys recent approaches to the repre- 
sentation of belief. There is a clash between the notion 
of degree of belief in the subjective probability tradi- 
tion and the idea of certainty as accepted belief, often 
couched in the language of logic, especially modal logic. 
The attempt to consider degrees of certainty finds its ori- 
gin in the works of Francis Bacon, often opposed to the 
one of Pascal. However, by reviewing some more recent 
trends in the representation of uncertainty, such as pos- 
sibility theory, ranking theory, evidence theory, and 
imprecise probability, one may argue that these novel 
approaches try to bridge the gap between the two tradi- 
tions, even if dropping some favorite properties on the 
way, such as the additivity of degrees of belief and the 
adjunction law for accepted beliefs. 


1. A Clash of Traditions 


There are two traditions for representing belief, one 
based on probability, one based on modal logic. The 
probability tradition goes back to Pascal and other sci- 
entists of his time, who tried to measure our confidence 
in events (today we also speak of “propositions” in the 
logical sense). In those times, a distinction was made 
between so-called chances and probabilities (Shafer, 
1978). The word “chance” was dedicated to repeatable 
events and measured via a combinatorial count of the 
frequencies of occurrence, assuming elementary events 
had equal chances to occur. The name “probability” was 
understood as a subjective notion, typically the confi- 
dence to be granted to a testimony. The latter view is still 
prevalent in important texts from the 18th century, such 
as d’Alembert and Diderot’s encyclopedia.’ The concept 
of subjective probability was to be taken up again in the 
20th century by scholars like Ramsey (1926/1931) and 
de Finetti (1937). They tried to justify why degrees of 
belief should be additive. However, additivity can only 
be obtained by introducing severe restrictions on the 


model of graded belief namely, it essentially prevents 
capturing the notion of ignorance. The reader is referred 
to the rather recent compilation by Huber and Schmidt- 
Petri (2009) for a survey of approaches to degrees of 
belief. 

In contrast, the modal logic tradition was developed 
in the 20th century, even if modal logic per se goes back 
to medieval philosophy and even to ancient Greek phi- 
losophy. Hintikka (1962) proposed a so-called epistemic 
logic that tried to define a rational logical framework for 
the idea of knowledge and its articulation with an all-or- 
nothing notion of belief. Namely, knowledge is essen- 
tially true belief, while a simply believed proposition is 
not supposed to be actually true. In this setting, mod- 
eling ignorance is obvious as you just express that you 
believe neither a proposition nor its negation. Besides, 
one essential feature of this approach is that when you 
know that each of two propositions is true, you also 
know that their conjunction is true, which is sometimes 
called the adjunction axiom (Kyburg, 1997). This axiom is 
also supposed to hold for beliefs, with the understand- 
ing that modal beliefs represent accepted beliefs, that is, 
propositions taken for granted to the point of reasoning 
with them as if they were true. 

There is a clash of intuitions between the two views, 
because for them to match, one needs to explain the 
connection between graded belief and belief simpliciter. 
One way is to add a threshold to graded beliefs and to 
retain propositions above this threshold, but it is then 
almost impossible to preserve the adjunction axiom 
if we model graded beliefs by probabilities. This is the 
point made by the lottery paradox (Kyburg, 1961), dis- 
cussed later in this chapter. 

The aim of this chapter is to provide an overview of 
belief representations that relate, and go beyond, sub- 
jective probability and epistemic logic, in the sense that 
some of them are graded versions of the epistemic logic 
approach and all of them give up the additivity assump- 
tion of probability theory. Most of these approaches have 
been developed only since the middle of the 20th century, 
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with a sudden acceleration from 1975 on. This chapter 
should be of particular interest to psychologists, because 
they often stick to the alternative “logic vs. probability” 
and are not always aware of the fact that there is a large 
area beyond. 


2. Rationality Axioms for Set Functions 
Representing Belief 


In this section, we survey mathematical models of belief 
and axioms that have been proposed to represent them. 
We consider the most general approaches and show that 
accepting the rule of adjunction considerably narrows 
the expressive power of representations of graded belief. 


2.1 Degrees of Belief 

We assume throughout the chapter that belief qualifies 
propositions, represented by subsets A, B, C, ... of a set 
S of possible worlds or states of affairs. More precisely, A 
stands for the proposition that event A occurs or equiva- 
lently that some entity of interest, say, the result x of a 
measurement, lies in A. We do not distinguish between 
logically equivalent propositions. Quantifying (the pos- 
sible lack of) belief is one instance of expressing uncer- 
tainty for an agent. 

The most usual representation of uncertainty consists 
in assigning to each proposition or event A a number 
g(A) in the unit interval [0, 1]. It evaluates the confidence 
(we deliberately use a more general term than “belief” or 
“plausibility”) of the agent in the truth of the proposi- 
tion “x EA.” This proposition can only be true or false by 
convention, even if the agent may ignore its truth-value. 
The following requirements sound natural: 


82) =9, — g(S) = 1, 


as you cannot have confidence in contradictions, and 
you should believe in tautologies. Monotonicity with 
respect to inclusion is also natural: 


If A CB then g(A) < g(B). 


Indeed, if A is more specific than B in the wide sense 
(in other words, A implies B), a rational agent should 
not be more confident in A than in B: all things being 
equal, the more imprecise a proposition, the more cer- 
tain it is. Under these properties, the function g is some- 
times called a capacity (after Choquet, 1953), sometimes 
a fuzzy measure (after Sugeno, 1977). In order to stick to 
the uncertainty framework, it was also called a confidence 
measure (Dubois & Prade, 1988). Such a set function rep- 
resents the epistemic state of an agent, that is, what an 
agent knows or thinks he or she knows (irrespective of 
whether it is true or not).” 
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An important special case of a confidence measure is 
the probability measure g = P, which satisfies the additiv- 
ity property: 

If ANB = ©, then P(A UB) = P(A) + P(B). 


Probability measures have very often been used to rep- 
resent degrees of belief, hence with a subjective flavor, 
as opposed to limits of relative frequencies of repeat- 
able events, which account for random phenomena. 
Contrary to frequencies, belief may be attached to 
unique events. However, the additivity axiom sounds 
much more natural when modeling frequencies than 
for degrees of belief. Actually, the latter were not sup- 
posed to be additive in pioneering works by Bernoulli 
and Lambert (Shafer, 1978). 

To recover additivity, scientists of the mid-20th century 
such as de Finetti (1937) interpreted P(A) as the amount 
of money the agent is ready to pay for buying a lottery 
ticket that earns $1 if the proposition A is true. Moreover, 
it is assumed that the price is fair, that is, the agent would 
accept to sell this ticket under the same conditions for 
the same price, and wants to avoid sure loss. This inter- 
pretation of degrees of belief entails the additivity axiom. 
Hence, to represent degrees of belief, probability measures 
should be used. Such probabilities are called “subjective” 
(see chapter 4.1 by Hajek & Staffel, this handbook). Bayes- 
ian probabilists consider nonadditive degrees of belief 
irrational, since they incur sure money loss. 

Remark: Supplying precise degrees of beliefs sounds 
cognitively very demanding. It seems more natural for 
an agent expressing his or her knowledge to only rep- 
resent the relative strength of confidence in various 
propositions, rather than trying to force him or her to 
deliver numerical evaluations. It is indeed easier to assert 
that one proposition is more credible than another than 
to assess a particular degree of belief (whose meaning is 
not always simple to grasp) or even to guess a frequency 
for each of them. The idea of representing uncertainty 
by means of partial order relations over a set of events 
dates back to Ramsey (1926/1931), de Finetti (1937), and 
Koopman (1940). They tried to find an ordinal coun- 
terpart to subjective probabilities. Later, philosophers of 
logic such as David Lewis (1973) have considered other 
types of relations, including comparative possibilities in 
the framework of modal logic. We omit discussing this 
literature for the sake of brevity. 


2.2 Accepted Belief 

Alternatively to measuring degrees of belief, one may 
want to consider propositions as beliefs accepted by an 
agent, if the latter is ready to reason as if such believed 
propositions were true. In particular, if A and B are 


Representing Belief: Beyond Probability and Logic 


beliefs, the conjunction ANB is also a belief (a debated 
issue). But the empty set should not be a belief. More- 
over, if A is a belief, and A CB, then B is a belief. In other 
words, a set of beliefs should be consistent and deduc- 
tively closed. 

Consider a Boolean-valued set function N from 2° to 
{0, 1} (here, 2° is the power set of S, i.e., the set of all subsets 
of S) such that N(A) = 1 if A is believed and = 0 other- 
wise. It is clear that N is a confidence measure (monotonic 
under inclusion) that satisfies the law of “minitivity”: 


N(ANB) = min(N(A), N(B)). 


This function is a special case of necessity measures 
(Dubois & Prade, 1988). It models the idea of certainty. It 
is easy to check that in the finite case, there exists a non- 
empty subset E =!) {A: N(A) = 1}, and that N(A) = 1 if and 
only if EGA. The set E contains all states the agent does 
not consider impossible, given his or her beliefs, which is 
the simplest representation of an agent’s epistemic state.” 
In other words, the function N describes propositions 
that can be proved from the epistemic state E. 


2.3 Extracting Beliefs from Confidence Measures 

In order to relate accepted beliefs to graded beliefs, one 
should extract accepted beliefs from a confidence func- 
tion g. A natural way of proceeding is to define a belief as 
a proposition in which an agent has enough confidence. 
So we should define a positive belief threshold 6 such 
that A is a belief if and only if g(A) = 6 > 0. However, the 
closure of accepted beliefs under conjunction entails the 
following property: 


Accepted belief postulate: If (A) = 6 and g(B) = B, then 
g(ANB) = B. 


This requirement is very strong. As the set of accepted 
beliefs should not include the empty set, it is clear that 
one should have min(g(A), g(B)) < B if ANB = @. Worse, 
if the postulate holds for any positive threshold £, then 
it is clear that g(AM B) = min(g(A), g(B)), but as g is mono- 
tonic under inclusion, it enforces the equality, which 
comes down to the statement that a capacity represents 
accepted beliefs if and only if g(AMB) = min(g(A), g(B)), 
that is, g is a graded necessity measure, still denoted by 
N. Letting 1, a mapping from S to [0, 1], be the function 
defined by u(s) = N(S\{s}) (the degree of belief that the actual 
state of affairs is not s), it is clear that N(A) = minseau(s). 
The value 1 - c(s) can be interpreted as the degree of plau- 
sibility m(s) of state s, where zis the membership function 
of a fuzzy epistemic state, usually called a “possibility dis- 
tribution” (Zadeh, 1978). The set function TT from 2° to 
[0,1] with 


TI(A) = 1- N(A‘) = maxsea (5), 
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where A‘ is the set-theoretical complement of A in S, rep- 
resents the degree of plausibility of A, measuring to what 
extent A is not ruled out by the agent. This setting is 
the one of possibility theory (Dubois & Prade, 1988). This 
means that possibility theory accounts for the notion of 
accepted belief. 

Possibility theory was proposed by Lotfi A. Zadeh 
(1978) in the late 1970s to represent uncertain pieces 
of information expressed by fuzzy linguistic statements 
and was later developed in an artificial intelligence per- 
spective (Dubois & Prade, 1988, 1998). Formally speak- 
ing, the proposal is quite similar to the one made almost 
30 years before by the economist George L. S. Shackle 
(1949), who advocated and developed a nonprobabilistic 
view of uncertainty based on the idea of degree of poten- 
tial surprise. The degree of potential surprise attached 
to proposition A can be modeled as N(A‘), namely, the 
more you believe A‘, the more surprising you find the 
occurrence of A. 

A necessity-like function was explicitly used by L. Jon- 
athan Cohen (1977) under the name “Baconian prob- 
ability.” It can be traced back to the English philosopher 
Francis Bacon, while probability was investigated by Pas- 
cal and followers, as pointed out by Cohen (1980). It is 
devoted to the idea of provability in contrast to prob- 
ability, and it perfectly fits necessity measures. Especially 
if you can prove A with some confidence, you cannot at 
the same time claim you can prove its negation, which 
makes the condition min(g(A), g(A‘)) = 0 natural. This 
condition is satisfied by necessity measures. So, the con- 
dition N(A) > 0 expresses that A is an accepted belief, the 
absolute value of N(A) expressing the strength of accep- 
tance. Such Baconian probabilities, viewed as grades of 
certainty, are claimed to be more natural than prob- 
abilities, for instance, for use in legal matters. Deciding 
whether someone is guilty cannot be done using statis- 
tics, nor can it be based on betting probabilities: you 
must prove guilt using convincing dedicated arguments. 

About a decade later, in the late 1980s, Wolfgang 
Spohn (1988) introduced the notion of ordinal condi- 
tional functions, now called ranking functions, as a basis 
for a dynamic theory of epistemic states. Ranking func- 
tions « are a variant of potential surprise, taking val- 
ues in the nonnegative integers, that is, N,(A‘S) = 2°” 
is a degree of potential surprise. The theory of ranking 
functions (see chapter 5.3 by Kern-Isberner, Skovgaard- 
Olsen, & Spohn, this handbook) can be developed in 
parallel with possibility theory,* even though they were 
independently devised. 

Subjective probability has been justified by Leon- 
ard Savage (1954) from first principles in the setting of 
decisions under uncertainty. This result has been very 
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important for popularizing probability as the natural 
way of representing degrees of belief. As it turns out, 
a similar approach has been carried out for possibility 
theory in the same act-based setting as the one of Sav- 
age, albeit assuming a finite state space S (for a detailed 
account, see Dubois, Prade, & Sabbadin, 2001). The 
approach leads to qualitative counterparts of expected 
utility and extends Wald’s pessimistic and optimistic cri- 
teria to possibility distributions. They may be considered 
not discriminating enough and can be refined using spe- 
cial forms of expected utility that encode lexicographic 
refinements of min and max (leximin and leximax; Far- 
gier & Sabbadin, 2005). For a survey, see Dubois, Fargier, 
Prade, and Sabbadin (2009); see also chapter 8.4 by Hill, 
this handbook. 


2.4 Probability and Accepted Beliefs 

It is clear that set functions representing accepted beliefs 
are at odds with probability measures supposed to cap- 
ture rational degrees of belief. If we apply the thresh- 
old method to a probability function in order to recover 
accepted beliefs, then we fail to satisfy conjunctive clo- 
sure since, however large the threshold £ is, 


P(A) > Band P(B) > 6 do not imply P(ANB) > B 
for all A, B. 


This point has been especially highlighted by Henry 
Kyburg (1961, 1997). He put forward the lottery paradox: 
if S contains the set of possible equiprobable outcomes 
of a chance game like buying a lottery ticket where only 
one wins, then, by making S big enough, we can make 
the probability of losing when betting on the chance 
outcome s, namely, (|S|-1)/|S|, arbitrarily high. But if 
you buy all tickets, you are sure to win. This example 
questions the cogency of accepted belief understood as 
high-enough degree of belief. As a consequence, Kyburg 
proposed to give up the idea that accepted beliefs are 
deductively closed and constructed an appropriate logic 
accounting for this standpoint (Kyburg & Teng, 2012). 

Another way out of the lottery paradox is suggested 
in Dubois, Fargier, and Prade (2004), namely, to restrict 
the set of probabilities. We look for probability measures 
that respect the adjunction rule, namely, for any context 
C and all A, B, 


P(A|C) > P(AS|C) and P(B|C) > P(B‘|C) 
imply P(ANB|C) > P(ASUB‘|C). 


This is the adjunction rule for 6 = 0.5. Such probabil- 
ity functions do exist, and they are called big-stepped 
probabilities (Benferhat, Dubois, & Prade, 1999) or also 
atomic bound probabilities (Snow, 1999); see also Leitgeb’s 
(2014) stability theory. They have probability masses of 
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the form pi>p2>---> Pn, where p; = P({s}), such that 
Pir Pisit Pinzt +++ +pn for all i<n, that is, they are the dis- 
crete counterpart of exponential probability functions. 
The same kind of result can be obtained for any accep- 
tance threshold. It suggests that the notion of accepted 
belief is more consistent with probability theory for dis- 
tributions that are strongly biased toward some specific 
outcome (the opposite of the uniform distributions used 
in the lottery paradox), that is, those for which common- 
sense beliefs can be entertained. 


3. Qualitative Possibility Theory: Reasoning 
with Defeasible Accepted Beliefs 


As mentioned before, there are two different traditions 
for doxastic reasoning: modal logic, which captures 
accepted beliefs by modalities, and set functions, which 
use probability. The fact that there is a class of set func- 
tions that also captures accepted beliefs suggests a bridge 
between the two traditions (Dubois et al., 2004). Let us 
also mention a third tradition, which models partial 
ignorance using a truth-functional many-valued logic 
(e.g., Kleene logic, which expresses partial knowledge 
about atomic propositions only). This severely limits 
its representation power (Ciucci & Dubois, 2013). We 
claim that qualitative possibility and necessity func- 
tions, valued in a bounded chain, offer a unified frame- 
work for these traditions. In particular, in this approach, 
accepted beliefs are defeasible, and it has close connec- 
tions with nonmonotonic reasoning, counterfactual 
reasoning, and belief revision (Dubois et al., 2004; see 
also chapter 5.2 by Rott and chapter 6.1 by Starr, both in 
this handbook). 


3.1 Boolean Set Functions and Modalities: 

A Simple Epistemic Logic 

Consider a propositional language Y where well-formed 
formulas p, q, . . . encode propositions. In artificial intel- 
ligence, a consistent set of formulas in propositional 
logic is often called a belief base and, if deductively 
closed, a belief set. In the syntax of propositional logic, 
you can express the fact that a proposition is believed, 
but there is no way to express that it is not believed. All 
you can express is that its negation is believed. Introduc- 
ing a belief modality 1 prefixing p improves the situ- 
ation as we can now distinguish between =Lp (p is not 
believed) and Lp (its negation is believed). The stan- 
dard approach to modeling accepted beliefs is indeed 
modal logic, after Hintikka (1962). Note that the mean- 
ing of the modality His described by the axioms ruling 
the logic. It can range from a very loose interpretation 
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(“the agent has received information that p is true”) to a 
very strong one (“the agent knows that p, in the sense of 
true belief that p”). 

The usual axioms of doxastic logic are those of the 
modal logic KD45 (see chapter 5.1 by van Ditmarsch, 
this handbook), which presupposes a modal language 
UM that extends ¥ and allows for nested modalities (the 
symbol = represents material implication): 


(PL) All axioms of propositional logics for M-formulas. 


(kK) (p= q) = (Cp= 9) 
(D) Up=-O-p 

(S4) Op=OUp 

(SS) -Op=UO-Op 


The inference rules are modus ponens and necessitation 
(if p is a theorem, deduce Lip). The semantics is in terms 
of accessibility relations R C Sx S. The satisfaction of Op 
at a state s based on relation R is defined by sR C [p], 
where SR = {s’: (s,s’) ER} and [p] denotes the set of states 
where p is true. 

This approach can be simplified in order to relate 
doxastic logic with the representation of accepted belief 
from the previous section: 


* KD45 uses a complex language, which we can restrict 
by putting the modality UH only before propositions 
in Z. 

* Axioms S4 and S5, which are often called “positive” 
and “negative introspection,” seem to be there basi- 
cally to make complex formulas of the language 
equivalent to simpler ones without nested modalities 
(even if they have a philosophical meaning). 


It may sound counterintuitive to evaluate doxastic for- 
mulas on a real state of affairs; it is more natural to do it 
on epistemic or doxastic states (nonempty subsets of S$). 

The propositional language ZY, whose atomic vari- 
ables are of the form Lp (p € Y), is the simplest language 
for an epistemic or doxastic logic. Note that it is disjoint 
from ¥ in the sense that it cannot express objective 
formulas. Then we only keep the axioms K and D, and 
necessitation is modeled by an axiom saying that Lip 
is valid whenever p is a tautology of the propositional 
calculus. This system is called MEL (minimal epistemic 
logic) (Banerjee & Dubois, 2014). 

The semantics of MEL is in terms of simple epistemic 
states ECS, and the satisfaction of Op by E is then 
expressed by E C [p], that is, p is true in all states that are 
not considered impossible for the agent, or alternatively, 
p is believed, that is, N([p]) = 1 for the Boolean neces- 
sity measure N equivalent to E. Indeed, axiom K ensures 
that K(p a q) is logically equivalent to Op a Lg, which is 
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the axiom of necessity measures. Axiom D expresses that 
N(A) = 1 implies TI(A) = 1. It ensures consistency (mod- 
els of MEL are then nonempty subsets of S). 

In other words, the logic MEL bridges the gap between 
epistemic logic (dropping the idea of introspection) and 
the representation of belief by means of set functions, 
identifying the latter with modalities. Especially the 
modality )p = -O-p corresponds to possibility measures. 


3.2 The Logic of Graded Acceptance 

The logic of acceptance MEL can be extended to graded 
necessity functions N using a simple multimodal logic. 
If L is a finite totally ordered scale of necessity degrees 
(understood as describing a gradation in accepted beliefs), 
we can expand the modal language of the MEL logic to 
allow for several belief modalities denoted by 11’, for AGL 
with A>0, where the sentence (1p encodes the statement 
N({p]) =A, and the 1-modality in MEL corresponds to 0! 
(expressing full belief). The language of this logic is thus 
a propositional language #" where atoms are of the form 
*y, pEL, for AGEL with A>0. Note that the formula 
=(’p stands for N([p]) < 4, which, due to the assumed 
finiteness of L, can be expressed as II([=p]) = 1-s(A), which 
is encoded as (=p, for the next lower value s(A) to A in 
the scale L. 

The sublogic obtained by fixing the value A is a 
copy of the logic MEL (it satisfies axiom K, axiom D, 
and necessitation). There is also the weakening axiom: 
“p= O"p if us A, and axiom D is valid in the stronger 
“y=>()'p. The semantics is in terms of L-valued 
possibility distributions a representing gradual epistemic 
states, and satisfies 1’p if and only if N([p]) = A, where 
Nis based on a. The soundness and completeness of this 
logic, called GPL (generalized possibilistic logic), have 
been proved (Dubois, Prade, & Schockaert, 2017). 

This logic is very expressive and enables one to reason 
about ignorance and defeasible beliefs. As shown in the 
previous reference, it can encode several nonmonotonic 
formalisms, especially: 


form 


* The older standard possibilistic logic (Dubois, Lang, 

& Prade, 1994) is obtained by restricting the language 
¥' to conjunctions of atomic epistemic statements 
“p, which are written (p,A) in the original syntax. 
This logic is nonmonotonic (Benferhat, Dubois, & 
Prade, 1998). 


* Conditional logics with statements of relative belief 
of the form N(p) > N(q) can be encoded by GPL for- 
mulas of the form Vj.0(C’p a 701’q). 

* The System P of Kraus, Lehmann, and Magidor (1990) 
uses nonmonotonic conditional statements p|-q. 


310 


They express the plausible inference of q from p, which 
is modeled by the constraint N(p= q) > N(p=74q). 

+ Answer-set programs can be expressed in GPL using 
a three-valued scale L = {1, A, 0}, where 1>A>0, and 
requiring two necessity modalities: a strong one, 0’, 


and a weak one, 1’. 


In summary, GPL can be viewed as the logic of qualita- 
tive Baconian probabilities. 


4. Nonadjunctive Settings for Rational Degrees 
of Belief 


If we take it for granted that belief should come in degrees, 
and if, like Kyburg, we reject the adjunction rule, we are left 
with nonclassical logics where deduction is not closed under 
conjunction, like for instance the logic of risky knowledge 
(Kyburg & Teng, 2012). Or we give up the Boolean frame- 
work of logic altogether and concentrate on the proper- 
ties of set functions that can model degrees of belief. The 
natural question is then whether degrees of belief should 
be additive at all. Fifty years ago, the answer was, Yes, of 
course. Since then, a number of proposals have emerged 
from which it follows that additivity should not be taken 
for granted. This section discusses reasons for questioning 
additive beliefs and focuses on some approaches to graded 
belief that are deliberately non-additive. 


4.1. Cana Single Probability Distribution Capture 

Any Epistemic State? 

The so-called Bayesian approach to subjective probabil- 
ity theory posits a uniqueness principle as a preamble to 
any kind of uncertainty modelling: Any state of knowl- 
edge is representable by a single probability distribution 
(see, for instance, Lindley, 1982). Note that indeed, if, 
following the fair bet procedure of de Finetti, an agent 
decides to directly assign subjective probabilities via 
buying prices to all possible outcomes in some game 
of chance, the coherence principle forces the agent to 
define a unique probability distribution. 

Yet another mathematical attempt to justify probabil- 
ity theory as the only reasonable belief measure is the 
one by Richard T. Cox (1946). He relied on the Boolean 
structure of the set of events and on a number of postu- 
lates he considered compelling. Let g(A |B) €[0,1] be a 
conditional degree of belief, A and B being events in a 
Boolean algebra, with B ©. The Cox axioms for condi- 
tional belief are as follows: 


(i) s(ANC|B) = Fg(A|CNB), g(C|B)) (if COB #2); 


(ii) g(AS|B) = n(g(A|B)) for B « ©, where A‘ is the com- 
plement of A in S; 
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(iii) The function F is supposed to be twice differentia- 
ble, with a continuous second derivative, while the 
function n is twice differentiable. 


Cox claimed that, on such a basis, g(A|B) must be iso- 
morphic to a conditional probability measure. 

This result has been repeated ad nauseam in the lit- 
erature on artificial intelligence to justify probabil- 
ity combined with Bayes’ rule as the only reasonable 
approach to represent and revise numerical degrees of 
belief (Horvitz, Heckerman, & Langlotz, 1986; Cheese- 
man, 1988; Jaynes, 2003). However, some reservations 
must be made. First, the original proof by Cox turned 
out to be faulty—see Paris (1994) for another proof, 
based on a weaker condition (iii) (it is enough that F 
be strictly monotonically increasing in each argument), 
but also on an additional technical density condition 
that requires an infinite setting. Moreover, Halpern 
(1999a,b) has shown that the result does not hold on 
finite spaces and that Cox’s original conditions do not 
suffice to prove the result in the infinite setting. Inde- 
pendently of these technical issues, it should be noticed 
that postulate (i) sounds natural only if one takes the 
form of Bayes conditioning for granted; and postulate 
(ii) requires self-duality, which rules out representations 
of uncertainty due to partial ignorance, as seen later on. 
The above comments seriously weaken the alleged uni- 
versality of Cox’s results. 

Applying the Bayesian credo as recalled above, jus- 
tified via the avoidance of Dutch books, assuming fair 
prices for bets, or by obedience to Cox’s axioms, forces 
the agent to use a single probability measure as the uni- 
versal tool for representing uncertainty, whatever its 
source. This stance leads to serious representation dif- 
ficulties already pointed out more than forty years ago 
(Shafer, 1976). For one, it means we give up making any 
difference between uncertainty due to incomplete infor- 
mation or ignorance, and uncertainty due to a purely 
random process, the next outcome of which cannot be 
predicted. One may indeed admit that additive degrees 
of beliefs are justified if they reflect extensive statistical 
evidence. But what if such information is not available? 

Take the example of die tossing: The uniform prob- 
ability assignment models the assumption that the die is 
fair. But if the agent assigns equal prices to bets assigned 
to all facets of the die, how can we interpret that? Is it 
because the agent is sure that the die is fair and its out- 
comes are driven by pure randomness (because, say, he 
or she could test it hundreds of times prior to placing the 
bets, or from counting cases)? Or is it because the agent 
who is given this die has just no idea whether the die is 
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fair or not, and so has no reason to put more money on 
one facet than on another? Clearly the epistemic state of 
the agent is not the same in the first situation as in the 
second one. But the uniformly distributed probability 
function is mute about this issue and handles the two 
situations in the same way. 

Next, the choice of a set of mutually exclusive out- 
comes depends on the chosen language, for example, 
the one used by the information source. However, sev- 
eral languages or points of view can co-exist for the same 
problem. Since there are several different possible repre- 
sentations of the state space, the probability assignment 
obtained from an agent will be language-dependent, 
especially in the case of ignorance: a uniform probabil- 
ity on one representation of the state space may conflict 
with a uniform one on another representation encoding 
of the same state space for the same problem, while in 
the case of ignorance this is the only available model. 
Shafer (1976) gives the following example: Consider the 
question of the existence of extraterrestrial life, about 
which the agent has no idea. If the variable v refers to 
the claim that life exists outside our planet (v = li), or 
not (v = ~li), then the agent proposes P,(li) = Pi(-li) = % 
on S; = {li, li}. However, it makes sense to interpret 
“life” as the disjunction between animal life (ali), and 
vegetable life only (vli), which leads to the state space 
S» = {ali, vli, -li}. The ignorant Bayesian agent is then 
bound to propose P>2(ali) = P2(vli) = P2(7li) = 1/3. As “li” 
is the disjunction of “ali” and “vli,” the distributions P; 
and P2 are not compatible with each other, while they are 
both supposed to represent ignorance. Another example 
comes from noticing that expressing ignorance about a 
real-valued quantity by means of a uniform distribution 
for x €[a, b], a positive interval, is not compatible with a 
uniform distribution on y = log(x) € [log(a), log(b)], while 
the agent has the same level of ignorance about x and y. 

Finally, Ellsberg’s (1961) paradox (see chapter 8.2 by 
Peterson, this handbook) showed quite early that, when 
expressing preferences between gambles consisting in 
drawing balls from an urn the content of which is ill 
known, many experiments have shown that people tend 
to systematically violate Savage’s axioms (especially the 
sure-thing principle), because they are pessimistic about 
rewarding events of unknown probability. One way of 
accounting for the results of these experiments is to give 
up the additivity of degrees of belief. 

The above limitations of the expressive power of sin- 
gle probability distributions have motivated the emer- 
gence of other approaches to representing uncertainty. 
Some of them, as seen above, give up the numerical set- 
ting of degrees of belief and use ordinal or qualitative 
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structures, like qualitative possibility theory. Another 
option is to tolerate incomplete information in the 
probabilistic approach, which leads to different math- 
ematical models of various levels of generality. These are 
reviewed in the rest of this chapter. 


4.2 Shafer Belief Functions and the Merging 

of Uncertain Testimonies 

The theory of evidence by Glenn Shafer (1976) can be 
viewed as a specific interpretation of Dempster’s (1967) 
upper-and-lower-probability framework for handling 
imprecise statistical information, or as the revival of the 
concept of probability (as opposed to chance) invented in 
the 17th century by Jakob Bernoulli, and later by Johann 
Heinrich Lambert, dealing with the problem of represent- 
ing and merging unreliable testimonies (Shafer, 1978). 
Shafer’s book is clearly in the latter tradition. 

The main issue is first to model unreliable testimo- 
nies. Suppose that a witness claims that proposition E 
is true but the receiver only partially believes this state- 
ment, considering that the witness is reliable with prob- 
ability p. So p can be viewed as the degree of belief of 
the receiver in proposition E due to the unreliability of 
the witness. The additivity issue is raised by the question, 
to what proposition should the complementary weight 
1-p be assigned? The regular probabilist would assign 
it to the negated proposition E‘. But if the testimony E 
is interpreted by the receiver as its negation, it means 
that the latter thinks this witness lies, that is, that he or 
she says E when knowing it to be false. There is another 
option, namely that due to the incompetence of the wit- 
ness, there is a probability 1- p that the testimony is just 
useless. In the latter case, the probability 1- p is assigned 
not to E‘, but to the whole of S, that is, to the state of 
ignorance. That is to say, with probability p the receiver 
knows that E is true and nothing else, and with prob- 
ability 1-p he/she knows nothing, which is modeled 
by a basic assignment m from the power set 2° to [0,1] 
such that m(E) = p and m(S) = 1- p. A belief function that 
models such a simple unreliable testimony E is called a 
“simple support function.” 

More generally, consider a process whose outcomes 
are set-valued (i.e., imprecise) and uncertain (there is 
a probability value attached to each outcome). This is 
modeled by a more general basic assignment m from 2° 
to [0, 1] such that m(@) =0 and X%,.,m(E) = 1. Epistemic 
states E with m(E) > 0 are called “focal sets.” The degree 
of belief Bel(A) in a proposition A and its dual plausibil- 
ity degree PI(A) are then defined by 


Bel(A) = S{m(E): ECA, E = 2}; 
PI(A) = S{m(E): ENA # 2}. 


312 


It is clear that the belief function Bel is non-additive, for 
example, Bel(A) + Bel(A‘) < 1, and the degree of plausibil- 
ity is PI(A) = 1-Bel(A‘). In the case of a simple support 
function, observe that when A ~ S, then Bel(A) =p if ECA, 
and Bel(A) = 0 otherwise. 

It is important to point out that belief functions gen- 
eralize probabilities (recovered when for all E, m(E) >0 
implies that E is a singleton), Boolean necessity measures 
(recovered when m(E) = 1 for some epistemic state E), and 
also graded necessity measures (recovered when focal 
sets are all nested; an example of this is a simple support 
function). A degree of belief Bel(A) clearly evaluates the 
probability of proving A from the available information. 
A plausibility degree PI(A) evaluates the probability that 
A is not logically incompatible with the available infor- 
mation. To use Cohen’s (1977) terminology, belief func- 
tions join the probable and the provable, or bring Pascal 
and Bacon together. But doing so, belief functions are no 
longer additive, nor do they respect adjunction. 

The major problem addressed by the 17th- and 18th- 
century pioneers is the merging of such testimonies. They 
proposed special cases of what is now known as Demp- 
ster’s rule of combination: Let m, and m2 be two mass 
assignments coming from independent sources. The 
result of the combination is a mass assignment 1m © m2 
defined by 


for all ACS, 

(m,@mz)(A) = (1/K) 3 {m1 (A1) « m2(A2): A = ALMA}, 
where K = 3{1m;(A1) « M2(A2): A1 NA2 = @} and 

(m,® mp) (@) = 0. 


The assignment ,®mz consists in intersecting any 
two overlapping focal sets, each coming from a distinct 
source, computing the probability of obtaining each 
subset A via such an intersection, and renormalizing the 
obtained mass assignment, as some pairs of focal sets 
may be conflicting. In the case of merging two simple 
support functions focusing on the same set E where 
Bel,(E) = m,(E) = p; and Bel2(E) = m2(E) = p2, the resulting 
belief in E is (Bel, ®Bel2)(E) = pi+p2—-pip2, which leads 
to a reinforcement of the belief in E, a result already 
suggested by the pioneers of belief functions in the 
17th and 18th century. This combination rule assumes 
that sources of information are independent, which 
makes the reinforcement effect plausible. 

In Shafer’s book, a major question was whether all 
belief functions can be expressed as the result of merging 
independent simple testimonies in the form of simple 
support functions. It turns out that only a subclass of 
belief functions, called separable, can be generated in 
this way. Later on, Smets (1995) tried to extend the 
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notion of a simple support function so as to cover all 
belief functions. An extensive presentation of the theory 
of evidence as a theory of rational belief is proposed by 
Haenni (2009). 

Finally, criteria for decision under uncertainty, where 
the latter is described by a belief function, are studied 
by Smets and Kennes (1994) and Jaffray (1989). The for- 
mer propose to define a so-called pignistic probability 
measure from a belief function (generalizing Laplace’s 
principle of insufficient reason), and apply the expected 
utility criterion. This probability measure coincides with 
the well-known Shapley value in game theory (Shapley, 
1953) and in some sense projects the provable onto the 
probable. Jaffray (1989) proposes and axiomatizes an 
extension of Hurwicz’s criterion. 


4.3 Imprecise Probabilities, Desirability, 

and Generalized Betting Theory 

The alternative approach to the modeling of degrees of 
belief consists in revisiting de Finetti’s approach to sub- 
jective probability, dropping the constraint that the price 
proposed by a gambler for buying a lottery ticket should 
be fair. This view was pioneered by Cedric Smith (1961), 
Peter M. Williams (1975), and Robin Giles (1982), and 
more extensively developed by Peter Walley (1991). 

In this approach, the agent offers buying prices for 
gambles. A gamble is a function f from S to the real line 
that expresses losses (f(s) < 0) or gains (f(s) > 0), depend- 
ing on the actual state of affairs s. The gamble associ- 
ated with a particular event is its characteristic function. 
The agent is not committed to selling such gambles at 
the same price as their buying price. The approach relies 
on so-called desirable gambles (Walley, 1991), which the 
agent would agree to buy for a positive price. The set of 
desirable gambles contains at least all positive gambles. 
Moreover, the sum of two desirable gambles is also 
considered desirable, and a desirable gamble remains 
desirable when multiplied by a positive constant. The 
maximal price at which the agent accepts to buy a 
gamble is the maximal value a such that f- a is desir- 
able. It is called the lower prevision of a gamble f. It can 
be shown that given a set of gambles ¢ and their lower 
previsions LP(f), there is a convex set PF of probabilities, 
called a credal set, such that LP(f) = inf{Ep(f): PEF} is 
the lower expectation of f; according to #, for all FEY, 
where E>(f) is the expectation of f with respect to prob- 
ability P. One important point is that any convex set of 
probabilities can be represented by lower previsions on 
some family of gambles. 

In this setting, the upper prevision UP(f) of a gamble f 
is provably equal to -LP(-f). The upper prevision UP(f) is 
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thus the minimal selling price of f. If the credal set attached 
to a set of gambles and its lower previsions is empty, then 
the proposal is inconsistent and the agent incurs a sure 
loss after buying and resolving these gambles. Avoiding 
sure loss means that UP(f) = LP(f) for all gambles f- 

Moreover, due to the interaction between gambles, it 
may be that the consistent buying prices proposed by 
the agent for gambles f€¥Y are too low and could be 
raised without altering the credal set. A set of buying 
prices bp(f) for fE¥ is said to be coherent if and only if 
LP(f) = bp(f) for all fEY. In other words, a set of buying 
prices for a set of gambles Y is coherent if and only if for 
any fE%, 


inf{Ep(f): Eo(f) = bp(f) for all FG} = bp(f). 


Under this approach the degree of belief in proposi- 
tion A is a coherent lower probability P,,(A) = inf{P(A): 
P&P} = LP(14), the lower prevision of its characteristic 
function, where P is the credal set induced by the lower 
prevision LP(f) on some gambles. 

Some remarks are in order to position this approach 
with respect to other approaches to rational degrees of 
belief: 


* The epistemic state of the agent is here represented 
by a credal set, but there is no ill-known probability 
inside. In particular, the interval [P,,(A), P*(A)], where 
P*(A) = 1-P,,(A, is not supposed to contain an ill- 
known subjective (nor an objective) probability of A. 
Just as for belief functions, degrees of belief are pre- 
cise and modelled by coherent lower probabilities. 

* Mathematically, belief functions are a special case of 
coherent lower probabilities. They are super-additive 
set functions at any order, while lower probabili- 
ties from any credal set only satisfy the inequality 
P,(A UB) = P,(A)+P,(B) when A and B are disjoint. 
In particular, the mass function recomputed from P,, 
instead of Bel in (3) (called the Moebius transform of 
P,,) exists, and is unique but not necessarily positive 
(Chateauneuf & Jaffray, 1989). 

+ An attempt to justify belief functions as the only 
rational approach to degrees of belief under a betting 
framework in the style of Walley (and not as unreli- 
able testimonies) was recently published by Kerkvliet 
and Meester (2018). 


The gamble approach leads to a decision rule that is 
specific to the imprecise-probability setting, namely, a 
gamble fis preferred to a gamble g if and only the gamble 
h=f-g is desirable, that is, if the lower expectation of 
this gamble with respect to the corresponding credal set 
P is positive. This yields a partial ordering on gambles. It 
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implies that LP(f) = LP(g). The latter inequality is a deci- 
sion rule that solves the Ellsberg paradox, in contrast 
with the decision rule LP(f-g) = 0 that satisfies Savage’s 
sure-thing principle (see also chapter 8.2 by Peterson, 
this handbook). 


4.4 Quantitative Possibility in the Setting 

of Imprecise Probability 

It is natural to reconsider graded possibility and neces- 
sity measures in the setting of belief functions and 
imprecise probabilities. In fact they are at the crossroads 
of all non-additive approaches to uncertainty and may 
be interpreted in various ways: 


* Necessity measures are a special case of belief func- 
tions. Their characteristic property is to have nested 
focal sets. In other words, they model coherent argu- 
ments in favor or disfavor of propositions. They are the 
only family of belief functions that obey the adjunc- 
tion rule. Note that the weaker Baconian condition 
min(Bel(A), Bel(A‘)) = 0 for all ACS corresponds to 
overlapping (consistent) focal sets. 

+ As a consequence, necessity measures also stand for 
coherent lower probabilities. However, they corre- 
spond to a very cautious type of betting behavior, 
such that if the buying price for gambling on A is pos- 
itive then the agent feels obliged to sell this gamble at 
the maximal price (Giles, 1982). 

* One may borrow the operational semantics of the 
Bayesians to derive personal possibility and necessity 
degrees. If we adopt the framework of belief functions 
for representing an agent’s knowledge and accept the 
idea that a belief function induces a pignistic prob- 
ability for making decisions, then we may reverse this 
process. Given a subjective probability reflecting fair 
prices of gambles corresponding to random events, 
one may look for the least informative belief func- 
tion that induces this subjective probability. It can be 
proved that it is always a necessity measure (Dubois, 
Prade, & Smets, 2008). 

* Necessity functions induced on the unit interval by a 
suitable transformation of a Spohn ranking function 
(see chapter 5.3 by Kern-Isberner et al., this handbook) 
have nothing to do with lower probabilities. Basically, 
as shown in Spohn (1990), they are more closely related 
to powers of infinitesimal probabilities, for which the 
additivity axiom degenerates in the minitivity axiom. 

* Yet another interpretation of possibility theory is in 
terms of likelihood. In statistical inference, given a 
parametric probabilistic model P(-|6) where 6 €® is 
the parameter of the model, the probability P(R| 6) 
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based on data set R is not the probability of 6 based 
on R, only its likelihood. It represents a looser degree 
lik(@) = P(R| @) of confidence in 6 for the observer hav- 
ing received evidence R. Advocates of the likelihood 
approach (Edwards, 1972) refuse to attach prior prob- 
abilities to values of 6, basically because this quantity 
is not observable and is just a model artefact. Rather, 
it is natural to try and define the likelihood lik(A) 
for any A CO from the values lik(@) for EA. It has 
been shown that the only meaningful definition is 
lik(A) = max{P(R| 0): @€A} (Dubois, Moral, & Prade, 
1997; Dubois, 2006). Hence, in the absence of prior 
probabilities, a likelihood function can be interpreted 
as a possibility measure. However, this kind of possi- 
bility measure is determined only up to a multiplica- 
tive constant, a specific feature that makes likelihood 
theory yet another kind of possibility theory. 


5. Conclusion 


This chapter has presented a survey of various approaches 
to the notion of belief, reflecting the progress made in 
the last 50 years. It seems that the frontal opposition 
between degrees of belief and accepted beliefs, that is, 
the Pascalian and Baconian traditions, may be allevi- 
ated to some extent if we give up the requirement that 
degrees of belief should be additive. There is a range of 
mathematical models standing between probability and 
modal logics, some of which retain the adjunction rule 
of Baconian probabilities. Some approaches blend the 
two traditions and are consistent with the requirement 
that you cannot at the same time believe in a proposi- 
tion and believe in its negation. The Baconian tradition 
also touches upon the issue of formal argumentation, on 
which there is an abundant literature today (see Haenni, 
2009, for its connection with Shafer belief functions). 
Argumentation can be viewed as a rational approach to 
handle inconsistency in reasoning due to conflicting 
pieces of information. One may argue that Baconian 
probabilities (in the form of, e.g., necessity functions, 
ranking functions, and the like) represent imprecise but 
conflict-free information, while ordinary probabilities 
capture precise but conflicting observations. The new 
theories of belief deal with both imprecise and conflict- 
ing information and seem to bridge the gap between the 
two traditions of belief representation. One may then 
consider belief in a more dynamical setting, where start- 
ing with more or less probable conflicting evidence one 
proceeds towards the provable via a suitable deliberation 
process involving argumentation. 
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Notes 


1. See the entry “Probabilité,” accredited to Benjamin de 
Langes de Lubiéres (1714-1790), in Encyclopédie, ou Dictionnaire 
raisonné des sciences, des arts et des métiers, by D. Diderot and 
J.-B. Le Rond d’Alembert (http://enccre.academie-sciences.fr 
/encyclopedie/page/v13-p403). 


2. Halpern (2003) calls set functions of this kind “plausibility 
measures,” not even assuming a total order on events (replacing 
[0, 1] by a partially ordered set). However, this terminology may 
lead to confusion with Shafer’s older plausibility functions (see 
section 4.2). 


3. Or doxastic state—we do not distinguish between the two 
in this chapter. 


4. Up to the presence or not of technical assumptions like 
well-ordering in the infinite setting. 
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Section 5 Belief Revision, Defeasible Reasoning, and Argumentation Theory 


5.1 Doxastic and Epistemic Logic 


Hans van Ditmarsch 


Summary 


We present modal-logical semantics for knowledge and 
belief, alternative semantics for knowledge, and also sys- 
tems with interaction between knowledge and belief. 
We then present common knowledge and common 
belief, as well as distributed knowledge. We conclude 
with topics involving change of knowledge and belief: 
public announcements, unsuccessful updates, knowabil- 
ity, growth of common knowledge, and resolving dis- 
tributed knowledge. 


1. Knowledge 


1.1 Knowledge and Ignorance 

Let p stand for the proposition “Paneer is rose water.” 
This is true. Amir, however, is uncertain whether it is 
true (maybe paneer is soft cheese?). If it is true, he con- 
siders it possible that it is true and that it is false. If false, 
he considers it possible that it is false and that it is true. 
“Considers possible” induces a binary indistinguishabil- 
ity relation between states of the world that is therefore 
obviously reflexive and symmetric and that is also tran- 
sitive (see below for exceptions involving vagueness). 
It is an equivalence relation. Figure 5.1.1b depicts the 
situation. 

If Amir and his friend Bala are both ignorant whether 
p, and they know this about each other (and so on), 
then a simple model for that situation again consists 
of two states for the different truth values of p and is 
such that both Amir and Bala cannot distinguish them: 
figure 5.1.1b. 

In that situation, both know what the other is uncer- 
tain about. It is easy to conceive of scenarios where this 
is not the case, for example, when Bala considers it pos- 
sible that Amir knows p, that he knows 7p, and that he is 
ignorant about p: figure 5.1.1d. This is known as higher- 
order uncertainty. 

Models for higher-order uncertainty can be arbitrarily 
large. Suppose that Amir and Bala both know a number 


and that the other’s number is one more or one less: the 
consecutive numbers riddle (Littlewood, 1953). An infinite 
chain of possibilities results: figure 5.1.1f (p now stands 
for “the number is even”). 


1.2 The Modal Logic of Knowledge 

The language of epistemic logic is that of propositional 
logic expanded with inductive clauses K,p for “Agent a 
knows proposition p.” The dual modality (K),p can be 
defined by abbreviation as ~K,7p and stands for “Agent 
a considers p possible.” The language of epistemic logic 
is interpreted on pointed Kripke models. A Kripke model 
consists of a domain of abstract objects called “states” or 
“worlds,” a set of binary relations R, (one for each agent a) 
on this domain, and valuations V, for atomic propositions 
p, specifying for each state whether atom p is true or false 
there. (In this simplified presentation, the names of states 
are their value of atom p, and we also use p for other than 
atomic propositions.) A pointed Kripke model has a des- 
ignated “actual” state. Formula K,p (for any proposition 
p) is true in state s of a model iff p is true in all states t of 
the model that are R,-accessible from s. On models inter- 
preting knowledge, the accessibility relations are assumed 
to be equivalence relations. We will list and discuss the 
principles of knowledge below in view of how rational 
they are. The logic of knowledge is called S5. 


1.3 Belief 

A difference between knowledge and belief is that knowl- 
edge is supposed to be correct, whereas belief may be 
mistaken. But a belief’s being true does not make it knowl- 
edge. The difference between knowledge and belief is a 
notoriously difficult question. It also involves evidence 
and justification, as well as Gettier examples (Gettier, 
1963). It will not be addressed here. 

The correctness of knowledge is formalized in the 
axiom K,p — p (T), which characterizes the reflexivity of 
R,. Reflexivity guarantees nonemptyness of the relation. 
Removing this axiom from the logic allows an accessi- 
bility relation R, to be empty, in which case beliefs are 
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Models for knowledge and belief. States are named with the valuations of atoms. Visual conventions for relations R, are transitivity, 


symmetry except for a-labeled edges with arrows at end and reflexivity except for states from which lead a-labeled edges with arrows at end. 


These conventions are sufficient to display $5 models and KD45 models unambiguously for a given number of agents. 


inconsistent. A more austere way of preserving consis- 
tency but removing correctness is merely to require that 
you consider possible what you believe. In the logical lan- 
guage, instead of an inductive clause for knowledge, we 
now have an inductive clause B,p for “Agent a believes 
p.” Formally, the above requirement is then B,p > (B)ap 
(D). The logic of belief is called KD45. 

Typical examples are when p is true but agent a 
believes it to be false (figure 5.1.1g), and where p is true, 
a mistakenly believes that a and b believe it to be false, 
whereas b believes that a and b correctly believe it to be 
true (figure 5.1.1h). 

Knowledge and belief in these modal $5 and KD45 
incarnations have become standard since Hintikka 
(1962) but have roots going back to medieval and clas- 
sical times. 


1.4 Rationality and Knowledge 
We now present the principles of knowledge by way 
of questioning their rationality. Except for the correct- 
ness of knowledge, they are also the principles of belief. 
There are no principles for multiagent interaction. We 
therefore omit the index a in Ka. 


T Kp—p. 
What you know is true (factivity or correctness of 
knowledge). 


If this fails, why not call the notion “belief”? Some 
approaches still call it knowledge: if the set of counter- 
examples, which may or may not include what is actu- 
ally the case, is sufficiently small (according to some 
formal notion of small and large, such as measure the- 
ory; Ben-David & Ben-Eliyahu, 1994), in other words, if 
the counterexamples are insignificant. Or it is the fal- 
lible knowledge of Ozgiin (2017, chapter 5), which ties the 
notion of knowledge only to justification and evidence. 


4 Kp—kkp. 
You know what you know (positive introspection). 


You have to make pairs of matching color out of the heap 
of socks coming out of the washing. Two socks may have 
the same color, and the next pair, and the next pair... 
But you are likely to end up with a last pair of socks of 
different color. The “same-color” relation is not transi- 
tive. Knowledge of socks’ colors is vague (van Deemter, 
2010). A different kind of attack emphasizes that ratio- 
nality is bounded. Birds know how to fly, but they do 
not know that they know how to fly. Higher-order cogni- 
tion in animals is (also) addressed in Verbrugge (2009). 


5 (K)p > KiKip. 
You know what you do not know (negative intro- 
spection). 


Negative introspection is most often attacked: I am 
unaware of much that I do not know. So I do not know 
what I do not know. Negative introspection is justifiable 
if the relevant facts that agents may be uncertain about 
are known: the closed-world assumption. The $5 notion 
of knowledge is considered suitable for describing the 
cognitive architecture of artificial agents. Well known is 
the interpreted system (Fagin, Halpern, Moses, & Vardi, 
1995) consisting of a set of global states composed in 
turn of the local states of individual agents/processors. 
An agent cannot distinguish global states with the same 
local state value: $5 knowledge. 

Various alternative modal logics of knowledge weaken 
negative introspection. The logic $4 is the system without 
5. Building a ladder from $4 to $5 are the systems $4.2, 
$4.3, and $4.4, obtained by adding the principles below 
to $4, for which (including 5) we also give corresponding 
relational properties. The .4 property is known as “remote 
symmetry.” 
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2 (K)Kp > K(K) p Vxyz (Rxy a Rxz > dw (Ryw a Rzw)) 
3 K(p(K)q) v Kq=(K)p) Vxyz (Rxy a, Rxz = Ryx v Rzy) 
‘4 p— (K)p > K(k) p) 
5 (K)p>KiK)p 
Negative introspection has also been criticized because 
in combination with factivity, it implies p—K(K)p 
(this characterizes symmetry; Hintikka, 1962; Stalnaker, 
2005). This direct connection between objective truth 
and subjective truth is considered unacceptable: the 
agent should not be “allowed” to create knowledge out 
of thin air. 

In logics combining knowledge and awareness (Fagin 
& Halpern, 1988; such logics are often known as logics 
for bounded rationality), the failure of negative intro- 
spection may even define unawareness. You are unaware 
of p if you don’t know p and you also don’t know that 
you don’t know p: -Kpa7K-Kp. This is known as the 
Modica-Rustichini definition of unawareness (Modica & 
Rustichini, 1994). 


Vxyz (Rxy a Ryz = Rzy) 
Vxyz (Rxy a Rxz > Ryz) 


K(p > q) > (Kp > ka) 
Deductive closure under the scope of knowledge. 


This principle and the next principle of “necessitation” 
are together often known as those embodying logical 
omniscience and thus full rationality. Logics satisfying 
weaker notions, of which we will give two examples, are 
associated with bounded rationality. 

In neighborhood structures (Pacuit, 2017), a formula 
is known in world w if there is a neighborhood of w (a des- 
ignated subset typically containing w) that is the exten- 
sion of the formula (i.e., the set of worlds where the 
formula holds). Let pq and p have such extensions. 
Their intersection is contained in the extension of q. If 
the latter is not a neighborhood, p—q and p are both 
known, but not q. 

In certain logics of knowledge and awareness (Fagin 
& Halpern, 1988; Halpern & Pucella, 2011), you know p 
if p is a member of the (fixed) set of “knowable” formu- 
las in every accessible state; p—>q and p may always be 
in that set, but not q. In that case, again, p> gq and p are 
both known, but not q. 


p implies Kp. 
Necessitation of knowledge (i.e., if p is valid, then Kp 
is valid). 


In neighborhood semantics, instead of necessitation, 
the weaker rule of monotony may hold: “p—q implies 
Kp — kq.” In logics of knowledge and awareness, necessi- 
tation may not hold either: p v =p is valid, but K(p v =p) 
is invalid if the agent is unaware of p. 
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1.5 Combining Knowledge and Belief 

Principles that are typically part of systems combining 
knowledge and belief, and where the main difference 
between the two is that beliefs, but not knowledge, may 
be incorrect, are Kp > Bp (knowledge implies belief) and 
Bp — BKp (you believe to know what you believe). 

The logics KD45 and $5 do not combine very well. If 
apart from Bp > BKp one also has Bp > KBp, then belief 
trivializes to knowledge: Bp <> Kp (van der Hoek, 1993). 

But KD45 and $4.4 combine well. In $4.4, belief is defin- 
able in terms of knowledge, namely, as Bp < (K)Kp, as was 
observed by Lenzen (1978). In this combination, knowl- 
edge is also defined by belief, namely, as Kp <= pa Bp. 
Halpern, Samet, and Segev (2009) call this explicit defin- 
ability. They also show that in the $5-KD45 combination, 
knowledge is not explicitly definable in terms of belief, 
but only implicitly. 


1.6 Knowing Whether 

You know whether p, formally Kw p, if you know that p or 
you know that ~p—more generally, if all accessible states 
have the same value for p. The principles of “knowing 
whether” are different from the principles of knowledge, 
for example, Kwp <> Kw ~p. For the expressivity of such 
epistemic logics, this does not matter, as “knowing that” 
and “knowing whether” are interdefinable: Kp <> pa Kwp, 
and Kwp <= Kp v K~p. In modal logics without the princi- 
ple Kp~p, such as KD45, knowledge may no longer be 
definable in terms of knowing whether. Indeed, the mod- 
els of figure 5.1.1a and figure 5.1.1g are indistinguish- 
able in the p states; in the former, Amir believes that p, and 
in the latter, Amir believes that -p; therefore, in either 
case, Amir “believes whether p” (in natural language this 
is infelicitous). Knowing-whether logics are founded in 
logics of contingency (Montgomery & Routley, 1966). 
Related notions such as knowing value, knowing how, 
and so on are already discussed in Hintikka (1962). A 
recent survey is Wang (2018). 


2. Common Knowledge and Belief, 
and Distributed Knowledge 


When celebrating Christmas with your young cousin, 
you should not spoil the fun by telling her that Santa 
Claus is actually dressed-up Uncle Ben. In fact, she 
knows this. So you both know this. But it is not com- 
mon knowledge. Only when she yells, “But it is Uncle 
Ben,” does it become common knowledge. Lewis (1969) 
gave the example of knowing that you have to drive 
on the left-hand side of the road: you know it, the car 
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approaching you on this narrow road knows it, but you 
would feel very unsafe if it were all you know. However, 
it is common knowledge that you have to drive on the 
left-hand side. But how can you have obtained common 
knowledge with the total stranger in the other car? This 
is absurd. This sort of common knowledge is a conven- 
tion: you are supposed to know. If you cause an accident, 
you cannot claim ignorance. 


2.1 The Modal Logic of Common Knowledge 

To the logic of knowledge, we add operators Cp for “The 
agents have common knowledge of p.” The dual opera- 
tor is (C)p. The formula (C)p is true in a state of a Kripke 
model iff p is true at the end of any finite path of acces- 
sibility/indistinguishability links involving any agent. 
This accessibility is described by the arbitrary iteration 
of the union of all accessibility relations: let Rc be the 
transitive closure of the union of all relations R,, then 
Cp is true in state s if p is true in all states t that are Rc- 
accessible. Similarly, we add to the logic of belief modali- 
ties CB p for common belief of p. 

Common knowledge has the properties of knowledge, 
but common belief does not have the properties of belief. 
It is easy to see that Cp—>p, Cp—>CCp, and (C)p—C(C)p. 
But for common belief, negative introspection does not 
hold. Figure 5.1.1g demonstrates that ~CBp—CB ~CBp 
is false. 

For example, in figure 5.1.1f, it is common knowledge 
that Bala does not know whether p. The logic of com- 
mon knowledge is indeed more expressive than the logic 
of knowledge. This is proved by comparing the set of all 
finite-length chains (with leftmost actual state) consist- 
ing of iterated paired b-a links with that same set plus the 
infinite chain as in figure 5.1.1f. The formula (C)(K),p can 
distinguish the former from the latter set, but no formula 
in the logic of knowledge can distinguish these sets, as 
it cannot look into the chains beyond the finite modal 
depth of that formula. 

The history of common knowledge comes with the 
names Friedell (1969) and Lewis (1969). Van Ditmarsch, 
van Eijck, and Verbrugge (2009) give a historical survey 
in the form of a Socratic dialogue. 


2.2 Relativized Common Knowledge 

Temporal logic with the binary “until” modality is more 
expressive than temporal logic with the unary “going to 
be” (always in the future) modality. Similarly, epistemic 
logic with the binary so-called relativized common knowl- 
edge modality is more expressive than that with the 
unary common knowledge modality (Kooi & van Bent- 
hem, 2004). In a given state s, the agents have common 
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knowledge of p relative to g (notation C,p) iff p is true in 
all states t that are reachable by finite Rc-paths satisfying 
q. Figure 5.1.1e illustrates the difference between relativ- 
ization and a “mere” antecedent in an implication: in 
the pq state, C,p is true, but C(q = p) is false. 


2.3 Distributed Knowledge 
Suppose that Amir knows that p and that Bala knows 
that p—q. Then together they know that q: if they were 
to communicate, they could make q common knowl- 
edge. Seen from a static perspective: if they were the 
same agent, then that agent could deduce q from p and 
pq. That agent only considers a world possible if Amir 
and Bala both consider it possible. We write Dp for “The 
agents have distributed knowledge of p”; Dp is true if p is 
true in all states accessible by the intersection of all acces- 
sibility relations R,. The notion of distributed belief goes 
back to Hayek (1945) and Halpern and Moses (1990). 
The logic with distributed knowledge is more expres- 
sive than the logic without: figure 5.1.1¢ and figure 5.1.1i 
cannot be distinguished in the logic with merely K, and 
K, (the models are bisimilar), but in the former, Dp is false 
in the p state, whereas in the latter, it is true in the p states. 
Asurvey of logics of knowledge, belief, common knowl- 
edge, and distributed knowledge is van Ditmarsch, Hal- 
pern, van der Hoek, and Kooi (2015, chapter 1). Goldblatt 
(2003) presents modal logics in their historical context. 


3. Change of Knowledge and Belief 


The formation of knowledge and belief is crucial to give 
meaning to the concepts of knowledge and belief, and 
even more so for common knowledge and belief (see also 
chapter 2.6 by van Benthem, Liu, & Smets, this handbook). 


3.1 Unsuccessful Update 

Moore (1942) observed the incoherence of the statement 
“T went to the movies last Tuesday and I don’t believe 
that.” This form, p a ~Kp (belief can be treated similarly), 
became later known as the Moore sentence. Assuming 
that you believe what you say, we get K(pa ~Kp). Using 
the discussed properties of knowledge, this is equivalent 
to Kpa K~Kp and thus to Kpa ~Kp, which is contradic- 
tory. Now consider Bala informing Amir that Amir does 
not know that p is true; by conversational implicature, 
this is formalized as p a ~K,p, which is not contradictory. 
(Hintikka [1962] devotes a whole chapter to such phe- 
nomena.) However, Bala cannot truthfully inform Amir 
twice in this way. After informing Amir of pa ~K,p, true 
at the moment of utterance, Amir has learnt that p, that 
is, Kap, so p a ~K,p is now false. 
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We can semantically verify this in public announcement 
logic (Plaza, 1989). A public announcement is a novel piece 
of information that is considered reliable (it is assumed 
true) and is simultaneously observed/incorporated by 
all agents. The result of the public announcement of p 
is the restriction of the Kripke model to the subdomain 
satisfying p. Now consider figure 5.1.1b: in the p state, 
pa7kyp is true. In the 7p state, it is not. This announce- 
ment therefore induces the transition of figure 5.1.1b to 
figure 5.1.1a, the singleton-p state, in which obviously 
K,p is true. The dynamic interpretation of the Moore sen- 
tence as an unsuccessful update is due to Gerbrandy (2007; 
see also van Ditmarsch & Kooi, 2006). Furthermore, the 
Moore sentence demonstrates that announcing a propo- 
sition need not make it common knowledge. 


3.2 Growth of Common Knowledge 

The other problem with common knowledge is that it 
does not exist. This is very clear in the case of complete 
strangers in approaching cars having “common knowl- 
edge” of the traffic rules but less clear in other cases. In 
order to obtain common knowledge, special observation 
conditions need to apply. The agents must simultaneously 
observe the event and must also have joint awareness of 
their observing this event. Such conditions are fulfilled in 
a group of people facing each other wherein one of them 
is addressing the group: the observation is of a sound 
under conditions of mutual eye contact. This explains 
the term “public announcement.” Loopholes can always 
be found: you may not have been paying attention, you 
could not hear what was being said, and so on. But you 
would then later need an excuse if you did not act upon 
the information: you were supposed to have heard. 

The directness of speech in company should be con- 
trasted with separating the sending and receiving of 
messages, after which growth of common knowledge is 
impossible (Halpern & Moses, 1990). 


3.3 Knowability 

In figure 5.1.1b, we can make two announcements, 
assuming the truth of p: the trivial announcement 
or the announcement of p. So, there is an announce- 
ment after which K,p, and there is an announcement 
after which (still) +K,p. Consider a logic with quantifi- 
cation over announcements: let @Kp mean that there is 
an announcement after which p is known. Inspired by 
Fine (1970), such a logic was proposed in Balbiani et al. 
(2008). We can also read Kp as “p is knowable,” such 
that p— Kp means that all truths are knowable. This is 
not valid. We have already seen that p a ~Kp is not know- 
able. Knowability (Fitch, 1963) is indeed interpreted in 
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such dynamic terms in van Benthem (2004) and Balbi- 
ani et al. (2008). A surprising result is that (Kp v K7p) 
(i.e., @Kwp) is valid (van Ditmarsch, van der Hoek, & 
lliev, 2012). 


3.4 Changing Knowledge and Belief Simultaneously 
We can also change $5 knowledge and KD45 belief simul- 
taneously. Plausibility models are Kripke models with 
equivalence relations R, to encode knowledge, but where 
the states in each epistemic equivalence class are ordered 
according to plausibility. The agent believes p iff p is true 
in all most plausible states (the order has to be a well- 
preorder, which guarantees that most plausible states 
do exist). This induces a doxastic relation R; to interpret 
belief, contained in R,. A public announcement restricts 
the epistemic relations to the novel domain. It is then 
determined anew what the most plausible states are, and 
this determines the new doxastic relation. 

Figure 5.1.1b and figure 5.1.1g jointly represent that 
Amir does not know whether p but believes that =p. After 
public announcement of p, the single remaining state is 
p: figure 5.1.1la. This p state has now become the most 
plausible state. So figure 5.1.1a also depicts the doxastic 
relation. Amir now knows and believes that p. He has 
revised his knowledge and his belief (see Baltag & Smets, 
2008; van Benthem, 2007; van Ditmarsch, 2005; a pre- 
cursor is Spohn, 1988. These matters are also discussed 
in chapter 5.2 by Rott and chapter 5.3 by Kern-Isberner, 
Skovgaard-Olsen, & Spohn, both in this handbook). 


3.5 Resolving Distributed Knowledge 

Not all distributed knowledge can become common 
knowledge. Let us give two very different examples. In 
the p state of figure 5.1.1b, it is distributed knowledge 
that p is true, and agent a does not know this: we have 
that K,(pa7K,p), and thus D(pa7K,p). But this cannot 
become common knowledge: if b informs a of p,p is 
common knowledge, not pa ~K,p. Second, consider fig- 
ure 5.1.1i: as we have already seen, p is distributed knowl- 
edge between a and b. But communication between a 
and b cannot make this common knowledge, because fig- 
ure 5.1.1i and figure 5.1.1b encode the same information 
in epistemic logic (they are bisimilar). Making distributed 
knowledge common knowledge is called resolving distrib- 
uted knowledge by Agotnes and Wang (2017). 


Acknowledgments 
Hans van Ditmarsch is also affiliated with IMSc (Chen- 


nai, India) and IrIsA (Rennes, France). Reviewers are 
kindly thanked for their comments. 


324 


References 


Agotnes, T., & Wang, Y. N. (2017). Resolving distributed knowl- 
edge. Artificial Intelligence, 252, 1-21. 


Balbiani, P., Baltag, A., van Ditmarsch, H., Herzig, A., Hoshi, T., 
& de Lima, T. (2008). “Knowable” as “known after an announce- 
ment.” Review of Symbolic Logic, 1(3), 305-334. 


Baltag, A., & Smets, S. (2008). A qualitative theory of dynamic 
interactive belief revision. In G. Bonanno, W. van der Hoek, & 
M. Wooldridge (Eds.), Logic and the foundations of game and deci- 
sion theory (LOFT 7) (Texts in Logic and Games, Vol. 3, pp. 13-60). 
Amsterdam, Netherlands: Amsterdam University Press. 


Ben-David, S., & Ben-Eliyahu, R. (1994). A modal logic for 
subjective default reasoning. In Proceedings, Ninth Annual IEEE 
Symposium on Logic in Computer Science (pp. 477-486). IEEE 
Computer Society Press. 


Fagin, R., & Halpern, J. Y. (1988). Belief, awareness, and limited 
reasoning. Artificial Intelligence, 34(1), 39-76. 


Fagin, R., Halpern, J. Y., Moses, Y., & Vardi, M. Y. (1995). Rea- 
soning about knowledge. Cambridge, MA: MIT Press. 


Fine, K. (1970). Propositional quantifiers in modal logic. Theo- 
ria, 36(3), 336-346. 


Fitch, F. B. (1963). A logical analysis of some value concepts. 
Journal of Symbolic Logic, 28(2), 135-142. 


Friedell, M. F. (1969). On the structure of shared awareness. 
Behavioral Science, 14, 28-39. 


Gerbrandy, J. (2007). The surprise examination in dynamic 
epistemic logic. Synthese, 155(1), 21-33. 


Gettier, E. (1963). Is justified true belief knowledge? Analysis, 
23, 121-123. 


Goldblatt, R. (2003). Mathematical modal logic: A view of its 
evolution. Journal of Applied Logic, 1(5-6), 309-392. 


Halpern, J. Y., & Moses, Y. (1990). Knowledge and common 
knowledge in a distributed environment. Journal of the ACM, 
37(3), 549-587. 


Halpern, J. Y., & Pucella, R. (2011). Dealing with logical omni- 
science: Expressiveness and pragmatics. Artificial Intelligence, 
175(1), 220-235. 


Halpern, J. Y., Samet, D., & Segev, E. (2009). Defining knowl- 
edge in terms of belief: The modal logic perspective. Review of 
Symbolic Logic, 2(3), 469-487. 


Hayek, F. (1945). The use of knowledge in society. American 
Economic Review, 35, 519-530. 


Hintikka, J. (1962). Knowledge and belief: An introduction to the 
logic of the two notions. Ithaca, NY: Cornell University Press. 


Kooi, B., & van Benthem, J. (2004). Reduction axioms for 
epistemic actions. In R. A. Schmidt, I. Pratt-Hartmann, M. 


Hans van Ditmarsch 


Reynolds, & H. Wansing (Eds.), AiML-2004: Advances in modal 
logic (Technical Report Series No. UMCS-04-9-1, pp. 197-211). 
Manchester, England: University of Manchester. 


Lenzen, W. (1978). Recent work in epistemic logic. Acta Philo- 
sophica Fennica, 30(2), 1-219. 


Lewis, D. K. (1969). Convention: A philosophical study. Cam- 
bridge, MA: Harvard University Press. 


Littlewood, J. E. (1953). A mathematician’s miscellany. London, 
England: Methuen. 


Modica, S., & Rustichini, A. (1994). Awareness and partitional 
information structures. Theory and Decision, 37, 107-124. 


Montgomery, H., & Routley, R. (1966). Contingency and non- 
contingency bases for normal modal logics. Logique et Analyse, 
9, 318-328. 


Moore, G. E. (1942). A reply to my critics. In P. Schilpp (Ed.), 
The philosophy of G. E. Moore (pp. 535-677). Evanston, IL: 
Northwestern University. 


Ozgiin, A. (2017). Evidence in epistemic logic: A topological per- 
spective (Doctoral thesis, University of Lorraine, Nancy, France). 


Pacuit, E. (2017). Neighborhood semantics for modal logic. Cham, 
Switzerland: Springer. 


Plaza, J. (1989). Logics of public communications. In M. L. 
Emrich, M. S. Pfeifer, M. Hadzikadic, & Z. W. Ras (Eds.), Pro- 
ceedings of the 4th International Symposium on Methodologies for 
Intelligent Systems (ISMIS 1989): Poster session program (pp. 201- 
216). Charlotte, NC: Oak Ridge National Laboratory. 


Spohn, W. (1988). Ordinal conditional functions: A dynamic 
theory of epistemic states. In W. L. Harper & B. Skyrms (Eds.), 
Causation in decision, belief change, and statistics (Vol. U, 
pp. 105-134). Dordrecht, Netherlands: Kluwer. 


Stalnaker, R. (2005). On logics of knowledge and belief. Philo- 
sophical Studies, 128(1), 169-199. 


van Benthem, J. (2004). What one may come to know. Analy- 
sis, 64(2), 95-105. 


van Benthem, J. (2007). Dynamic logic of belief revision. Jour- 
nal of Applied Non-Classical Logics, 17(2), 129-155. 


van Deemter, K. (2010). Not exactly: In praise of vagueness. 
Oxford, England: Oxford University Press. 


van der Hoek, W. (1993). Systems for knowledge and beliefs. 
Journal of Logic and Computation, 3(2), 173-195. 


van Ditmarsch, H. (2005). Prolegomena to dynamic logic for 
belief revision. Synthese, 147, 229-275. 


van Ditmarsch, H., Halpern, J. Y., van der Hoek, W., & Kooi, 
B. (Eds.). (2015). Handbook of epistemic logic. London, England: 
College Publications. 


van Ditmarsch, H., & Kooi, B. (2006). The secret of my success. 
Synthese, 151, 201-232. 


Doxastic and Epistemic Logic 


van Ditmarsch, H., van der Hoek, W., & Iliev, P. (2012). Every- 
thing is knowable—how to get to know whether a proposition 
is true. Theoria, 78(2), 93-114. 


van Ditmarsch, H., van Eijck, J. & Verbrugge, R. (2009). 
Common knowledge and common belief. In J. van Eijck & 
R. Verbrugge (Eds.), Discourses on social software (pp. 99-122). 
Amsterdam, Netherlands: Amsterdam University Press. 


Verbrugge, R. (2009). Logic and social cognition: The facts 
matter, and so do computational models. Journal of Philosophi- 
cal Logic, 38(6), 649-680. 


Wang, Y. (2018). Beyond knowing that: A new generation of 
epistemic logics. In H. van Ditmarsch & G. Sandu (Eds.), Jaakko 
Hintikka on knowledge and game-theoretical semantics (Outstand- 
ing Contributions to Logic, Vol. 12, pp. 499-533). Cham, Swit- 
zerland: Springer. 


325 


5.2 Belief Revision 


Hans Rott 


Summary 


This chapter gives an introduction to the problem of 
rational belief change and its formal modeling in quali- 
tative logical theories. It first presents an outline of the 
approach based on rationality postulates and of the most 
important constructive approaches within the classical 
AGM model of belief revision. Then it gives the basic 
ideas of extensions of the classical model to iterated belief 
revision, to multiple revision and to two-dimensional 
belief change, to belief merging and to belief updates, as 
well as to default reasoning. 


1. The Problem of Belief Revision 


The term “belief revision” refers to the change of a sys- 
tem of beliefs or opinions in response to new informa- 
tion, particularly to information that is inconsistent 
with this system. In the 1960s and 1970s, the works 
of Isaac Levi and William Harper prepared the ground 
for systematic studies of belief change processes. In the 
1980s, a seminal research paradigm of belief revision was 
established by Carlos Alchourrén, Peter Gardenfors, and 
David Makinson (also known as “AGM”), who discov- 
ered a common structure in the logic of normative sys- 
tems and of the logic of counterfactual conditionals (see 
chapter 6.1 by Starr, this handbook)—areas that turned 
out to be structurally related and were subsequently 
merged into the field of belief revision (also known as 
“theory change”). Belief revision theories are different 
from descriptive psychological investigations (see chap- 
ter 5.4 by Gazzo Castafieda & Knauff, this handbook) 
and investigations of theory dynamics as conducted in 
the philosophy of science in that they aim at providing 
postulates and construction recipes for rational processes 
of belief change. 

The problem of belief revision is best illustrated by an 
example. Suppose you believe that 


(a) The magician’s swords are real swords. 


(b) Lisa, the magician’s assistant, is in the box. b 
(c) If Lisa is in the box, she cannot escape the 

swords. b—>7e 
(d) If the magician’s swords are realswords sa-e—-d 


and Lisa cannot escape them, she will die. 


Like any person capable of good reasoning, you con- 
clude from these premises that 


(e) Lisa will die. d 


At the end of the show, however, Lisa climbs out of 
the box without a scratch. This is a surprise to you. You 
receive new information ~d that contradicts your initial 
premises (a)-(d). Thinking about the matter, you realize 
that given your initial premises, logic in a way compelled 
you to believe d. Now that you have found out, luckily, 
that d is not true after all, and you want to add ~d to 
your stock of beliefs, there are only two options: either 
your logical reasoning that delivered the conclusion d was 
faulty, or (at least) one of the premises (a)-(d) was false. 
Assuming that your logic has been flawless and you want 
to accept the new piece of information ~d, you have to 
give up (at least) one of your premises. Logic, however, is 
completely silent about which of the premises to give up. 
Something else has to step in that provides you with guid- 
ance how to proceed. Belief revision theories specify what 
this “something else” is and how it helps you to resolve 
such problems in the transformation of belief states. 

In general, you will not want to give up all of your 
beliefs in order to accommodate the new information 
>d and avoid becoming inconsistent. This radical move 
would mean an unnecessary loss of valuable informa- 
tion. So when giving up the belief d, you have to decide 
which of your reasons for holding d to retain and which 
to retract. You have to choose between retracting either s 
or b or b> 7e or sa ~e—d. You may, for instance, decide 
that you were too rash in endorsing b— =e. Lisa might 
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twist her body into a corner of the box so as to avoid 
being impaled by the swords. 


2. The Representation of Belief 


Any theory of the rational formation and transforma- 
tion of belief states consists of (1) a static part describ- 
ing the states of belief that a person may be in and (2) a 
dynamic part describing how belief states should change 
in response to external input. The static picture constrains 
the dynamic picture. Assuming that there are such things 
as plain beliefs, belief revision theorists need to decide 
how to represent beliefs (what a person believes) and belief 
states (the mental states that believers are in). 

This chapter considers qualitative models in which 
belief states don’t carry any numerical information and 
reasoners accept new information without numerical 
qualification. There are no numbers representing degrees 
of “certainty,” “security,” “plausibility,” and so on, and 
belief is always plain belief. In quantitative theories, on 
the other hand, reasoners do not, or in any case do not 
have to, accept new information simpliciter but can accept 
it with a certain degree of certainty or plausibility. The 
best-known quantitative modeling of belief is the proba- 
bilistic one, but there are alternative numerical methods 
of belief change using ranking functions (Spohn, 2012; 
similar ideas are part of possibility theory as summa- 
rized by Dubois & Prade, 2015; cf. chapter 4.7 by Dubois 
& Prade and chapter 5.3 by Kern-Isberner, Skovgaard- 
Olsen, & Spohn, both in this handbook). Quantitative 
approaches are more expressive than qualitative meth- 
ods. Their additional expressive power is a substantial 
advantage (it is essential, for instance, for expressing 
significant independence relations between beliefs), but 
there is a price to be paid for it: often it is not clear where 
the numbers come from and what exactly they are sup- 
posed to mean. While in probability theory, numerical 
values may be explicated in terms of betting quotients, 
the meaning of the numbers in ranking functions is less 
clear (but see Spohn, 2012, chapter 8). 

Belief revision theories have traditionally represented 
beliefs as sentences or propositions (i.e., roughly, contents 
of sentences). We will focus on sentential models that typ- 
ically work with a language L closed under applications of 
the Boolean operators of negation, conjunction, disjunc- 
tion, and material implication. 

There is an important contrast in modern epistemol- 
ogy between foundationalist and coherentist theories of 
the justification of belief. With a grain of salt, this con- 
trast is mirrored in belief revision theories. Foundational- 
ists in belief revision assume that there is a set of basic 
beliefs that are somehow given and may be thought of as 
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justified in a direct, noninferential way. All other beliefs 
are justified only insofar as they can be derived from the 
stock of basic beliefs. In more formal terms, such theo- 
ries presume that there is a belief base H of distinguished 
sentences that have, epistemologically speaking, an inde- 
pendent standing. Another factor to be decided in a for- 
mal modeling of belief states is the logic governing the 
language L. A logic or consequence operation is a function 
Cn that assigns to any belief base H the set Cn(H) of all 
logical consequences of H. It is commonly assumed that 
Cn is Tarskian and includes classical propositional logic. 

We call H a base for the belief set K if and only if 
K=Cn(A). Belief sets are sets K of sentences in L that are 
closed under Cn, that is, K = Cn(K). If A is an element of K, 
then A is believed (held to be true) by a reasoner whose belief 
set is K. If the negation 7A is an element of K, then A is 
rejected in K. Of course, in general, there are also sentences 
A that are neither believed nor rejected but on which the 
reasoner suspends judgment. A belief set can be viewed as 
a theory that is a partial description of the world. 

A belief derived from a belief base is justified by those 
elements of the base that are used in its derivation. Like 
other foundationalist theories in epistemology, the 
modeling using belief bases suggests that propositions 
should not be accepted as beliefs unless they are posi- 
tively justified. Reasoners are supposed to keep track of 
the justifications for their beliefs. In contrast, coherence 
theories hold that reasoners need not keep track of the 
pedigrees of their beliefs. They should instead focus on 
the logical or inferential structure of the beliefs—what 
matters for the rationality of a belief is how it coheres 
with the other beliefs that are accepted in the current 
belief state. For the coherentist, there is no designated 
set of basic beliefs: each belief depends in some way or 
other on there being certain other beliefs that support it. 

As models of explicit (active, occurrent) belief, bases are 
psychologically more realistic since they are usually finite 
entities. On the other hand, belief set dynamics offers a com- 
petence model that helps us to understand what people 
ought to do ideally if they were not bounded by limited 
logical or computational reasoning capabilities. There are 
also interpretations that make sense of the requirement of 
logical closure even for less-than-ideal reasoners: a belief 
set may be taken to represent the set of beliefs a reasoner is 
committed to (Levi, 1997) or the set of beliefs ascribed to a 
reasoner. More thoughts on logical closure can be found in 
chapter 3.1 by Steinberger (this handbook). 


3. Kinds of Belief Change 


If belief states are represented by sets of sentences, there 
can only be three doxastic attitudes: a sentence can be 
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accepted, rejected, or neither. We distinguish two basic 
types of belief change: a sentence A can be inserted into a 
belief set (turned into a belief) or deleted from a belief set 
(turned into a nonbelief). The former is called a revision 
by A, denoted by KA, and the latter a contraction with 
respect to A, denoted by K+A. 

An expansion K+A of a belief set K by a new piece 
of information A is simply formed by set-theoretic addi- 
tion and subsequent logical closure: K+A = Cn(K U {A}). 
An expansion thus defined is closed under logical con- 
sequence, and it is consistent as long as A is consistent 
with K. But what if A is inconsistent with K? According 
to the classical rule of ex falso quodlibet, a single inconsis- 
tency entails any arbitrary sentence, so there is only one 
inconsistent set closed under Cn, namely, the set L of all 
sentences in the language L. Thus, the expansion opera- 
tion is useless in the belief-contravening case. If consis- 
tency is to be preserved, belief-contravening changes 
require choices in retractions that cannot be made on 
the basis of set theory and logic alone. 

This specific problem of belief-contravening revision 
was illustrated by our introductory example. From a log- 
ical point of view, there were several ways of construct- 
ing the revision when accommodating ~d. There is no 
purely logical reason for making one choice rather than 
another among the sentences eligible for retraction. The 
reasoner needs additional information concerning these 
sentences and a well-defined method for constructing 
revisions that takes into account a number of coherence 
constraints (see section 4.1). Such a method is repre- 
sented formally by a revision function * associated with 
the reasoner’s belief set K that can take any sentence A 
and return as value the revised belief set K*A.' 

The contraction process faces parallel problems. The 
belief set K of our initial example contains the prem- 
ises s, b, b> -e, and sa 7e—d together with all their 
logical consequences (among which is d). Suppose that 
you want to contract K with respect to d. Of course, the 
target sentence d itself must be deleted from K when 
forming the contracted belief set K+d. But this is not 
enough. At least one of the four premises must be given 
up as well in order to prevent d from being rederived 
through Cn. Again, there is no purely logical reason for 
making one choice rather than another. Deleting d from 
your belief set poses quite the same problems as adding 
—d to it. 

We have now formed an idea that the problem of 
belief revision (by ~d) is closely related to the problem 
of belief contraction (with respect to d). In parallel with 
revision functions, one can introduce the concept of a 
contraction function + associated with the reasoner’s belief 
set K that takes sentences A and returns as values the 
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contracted belief sets K+A. While for belief-contravening 
revision, the problem of maintaining consistency is most 
palpable, in belief contraction, there is no problem with 
consistency: shrinking one’s set of beliefs can never 
introduce an inconsistency. But instead, the problem of 
logical closure makes itself felt rather acutely. The logi- 
cal interaction between an updated “data base” and its 
derived consequences may be seen as the ultimate source 
of the problem of belief change. 


4. Two Strategies for Characterizing 
Rational Changes of Belief 


When tackling the problem of belief revision in more 
concrete detail, two general strategies have been followed. 
First, one can write down a list of desiderata that an 
appropriate belief revision function should fulfill. That is, 
the standards for revision and contraction functions can 
be laid down in the form of rationality postulates. Second, 
one can present explicit constructions for rational changes 
of belief. The solution to the problem of belief revision 
will not be complete unless we know how to define and 
compute appropriate revision and contraction functions 
for a given belief state. We will see that constructions of 
belief changes make essential use of “doxastic preference 
relations” (or similar structures on which the necessary 
choices may be based). 


4.1 The Strategy Based on Rationality Postulates 

In their seminal study, Alchourr6n, Gardenfors, and 
Makinson (1985) proposed a set of general postulates for 
the rational revision of belief sets. They are now usually 
called the AGM postulates: 


(#1) 
(#2) 
(#3) 
(#4) 
(#5) 
(#6) 
(#7) 
(#8) 


KA is closed under Cn. 

AEK#A, 

KsxACK+A. 

If -A€K, then K+ACK*#A. 

If A is consistent, so is KA. 

If Cn(A) = Cn(B), then K*#A = K«B. 
K«(AaB) C (K*A) +B. 

If -B¢ KA, then (K*A)+BCK*(AnB). 


We can think of this set as coming in four pairs. Postu- 
lates (*2) and (#6) concern the input: the input should 
be accepted, and it is its content, not its syntactic form, 
that matters. The other three pairs encode three differ- 
ent ideas of coherence. Postulates (+1) and («5) represent 
a static notion of coherence very much like the idea of 
a reflective equilibrium. They concern the beliefs held 
by a person at a certain point in time. Consistency is 
the major driving force behind belief revision theories, 
and it is a minimal notion of static coherence.” Closure 
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is amore demanding idea of static coherence. Postulates 
(«3) and (*4) formulate a dynamic notion of coherence. 
They refer to a short sequence of belief states, namely, 
the transition from a state to its successor state—but 
only when the new piece of information A is consistent 
with the belief set K: in this case, the revision is formed 
by set-theoretically adding A to K and then taking the 
logical closure. Dynamic coherence in this sense instan- 
tiates a (restricted) maxim of conservatism or, in differ- 
ent terms, of informational economy, minimal change, 
minimum mutilation (Quine), or cognitive inertia. Prior 
beliefs should not be abandoned, and new beliefs should 
not be adopted, beyond necessity. Minimal change has 
frequently been advertised as one of the principal ideas 
motivating belief revision theories, but this claim is con- 
troversial (Rott, 2000). Postulates («7) and (*8) encode a 
dispositional notion of coherence referring to potential 
transitions from a single belief state to various possible 
successor states. More specifically, («7) and (#8) relate 
the change by a conjunction to the change by one of the 
conjoined sentences. Dispositional coherence in belief 
revision turned out to be closely related to the coher- 
ence of choices as studied in rational choice theory. 
While static coherence is essentially a logical notion, 
dynamic and dispositional coherence are notions relat- 
ing to two different concepts of economic behavior.’ But 
notice that all postulates except (*2) refer to the under- 
lying logic Cn. 

Isaac Levi advanced the thesis that any rational 
belief revision occasioned by a new piece of infor- 
mation A is decomposable into two successive steps, 
namely, an elimination of 7A, followed by an expan- 
sion by A (which produces no inconsistency, thanks 
to the antecedent contraction step). This suggestion, 
which is in accordance with the intuitive discussion of 
our introductory example, has become known as the 
Levi identity: 


(L) K*A = (K+7A)+A. 


Many authors have endorsed (L) and sought to reduce 
the problem of belief revision to the problem of belief 
contraction. But there is also a converse equation defin- 
ing contraction in terms of revision. This is the so-called 
Harper identity, which is, in a rather precise sense, the 
converse of the Levi identity: 


(H) K+A = KN (K*7A). 


According to (H), the contraction of a belief set K with 
respect to A is that part of K that will survive K’s revision 
by 7A. Using equations (L) and (H) as bridge principles, 
a set of eight AGM postulates for the rational contraction 
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of belief sets is equivalent to the set of postulates for 
belief revision. Here are just the most interesting ones: 


(7) K+ANK+BC K+(AaB). 
(<8) IfA¢K+(AnB), then K+(AnB) CK=A. 


The conjunction postulates (+7) and (+8) compare 
contractions of K with respect to a conjunction AaB 
with contractions with respect to the conjoined sen- 
tences A and B. (+7) states that any sentence that is con- 
tained both in the contraction of K with respect to A and 
in the contraction of K with respect to B is contained in 
the contraction of K with respect to Aa B. As a kind of 
converse, if the contraction with respect to Aa B elimi- 
nates A, then any sentence contained in the contraction 
with respect to A a B is also contained in the contraction 
with respect to A. This is natural if eliminating A» B is 
taken to mean eliminating A or eliminating B (or elimi- 
nating both, if A and B are equally entrenched). 

AGM called the first six postulates for revisions and 
contractions the basic postulates and the last two ones 
supplementary postulates. The latter pair of postulates 
entails that belief change operations can be construed 
as being guided by some well-behaved doxastic prefer- 
ence relation (cf. section 4.2). In many contexts, how- 
ever, postulates (*7) and (#8) have turned out to be too 
strong. Interesting weakenings, for example, are the 
following: 


(*7c) If BEKx«A, then K*(Aa B) C K#A. 
(*8c) If BEK*A, then K*A C K*(A a B). 


Taken together, these conditions say that if B is con- 
tained in K*A, then K*A is equal to K*(Aa B). The “c” 
in “(#7c)” and “(*8c)” means “cumulative,” due to the 
fact that there are corresponding postulates for non- 
monotonic logics that are characteristic for cumulative 
reasoning (cf. section 6.4). 


4.2 The Constructive Strategy 

We have already seen that logical postulates are not suf- 
ficient to determine a particular change of K with respect 
to some proposition A. The construction of belief changes 
has to draw on extra-logical factors. In some construc- 
tions, the syntactic structure of the base for K is itself 
taken to encode the relevant information (Hansson, 
1991; Lewis, 1981; Nebel, 1992). Since the 1990s, most 
researchers have assumed that these extra-logical factors 
are doxastic preference relations (plausibility relations) over 
sentences or over possible worlds and that such relations 
are part of the reasoner’s belief state. Others have sug- 
gested that external, objective criteria (like objective prob- 
ability, informational content, truthlikeness, or objective 
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similarity between possible worlds) should play the key 
role in fixing the revision-guiding structures. 

AGM explored three methods for the construction 
of the contractions.’ The first method defines K+A as 
the intersection of the best maximal subsets of K not 
implying A, where what is best is determined by a dox- 
astic preference relation (partial meet contraction). This 
method was later shown to be intimately related to a 
semantic modeling using possible worlds (Grove, 1988; 
Katsuno & Mendelzon, 1991). Here, K+A is defined as 
the set of sentences in K that are true in all maximally 
“plausible” worlds that do not satisfy A, where what is 
most plausible is determined by a system of spheres of 
possible worlds centered on the worlds satisfying K. The 
second method keeps in K+A precisely those proposi- 
tions of K that are “safe” in all minimal subsets of K that 
imply A, where again what is safe is determined by a 
doxastic preference relation (safe contraction). The third 
method keeps in K+A, roughly speaking, those propo- 
sitions that are doxastically “well entrenched,” where 
entrenchment relations are yet another kind of doxas- 
tic preferences (entrenchment-based contraction). Impor- 
tantly, the doxastic preference relations used in each of 
these constructions are independent of the particular 
sentence to be contracted. 

It is natural to ask how the postulates strategy and 
the constructions strategy relate to one another. Assum- 
ing that doxastic preferences meet a number of formal 
requirements (like transitivity and completeness), AGM 
and their followers were able to prove various represen- 
tation theorems to the effect that a contraction operation 
for a belief set K satisfies the rationality postulates for 
contraction if and only if it can be (re)constructed by 
any of the standard construction recipes. Revisions are 
taken care of with the help of the Levi identity. As a cor- 
ollary, the three standard methods are thereby proven 
equivalent.° 

While the idea of informational economy has played 
a rather modest role in AGM-style theorizing of belief 
revision, ideas from economics have always formed an 
essential part of belief revision theories through the sys- 
tematic study of the structure and application of pref- 
erence relations to cognitive choice problems. Varying 
(usually weakening) the formal requirements for the rel- 
evant preference relations results in variations (usually 
weakenings) of the postulates encoding dispositional 
coherence. Some natural relaxations of the requirements 
for preference relations (including the renunciation of 
completeness) lead to the postulates (*7c) and (*8c) 
mentioned above. Rationality constraints for choices 
(as studied by leading economists like P. Samuelson, 
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K. Arrow, and A. Sen; see chapter 8.1 by Grtine-Yanoff, 
this handbook) can be applied in quite a straightfor- 
ward way to choice problems involved in belief change 
processes—choices concerning least plausible beliefs or 
concerning most plausible possible worlds—and were 
thus shown to “translate” into rationality postulates for 
belief revision (Rott, 2001). Seen from this perspective, 
the problem of belief revision is just an instantiation 
of the general problem of rational choice, with a few 
additional constraints placed by the background logic 
Cn. While the formation and transformation of belief 
sets would usually be classed as a problem of theoreti- 
cal reason, the making of choices belongs to the realm 
of practical reason (cf. chapter 2.2 by Wedgwood, this 
handbook). However, the interpretation of this connec- 
tion is far from clear. Should we take the use of choice 
functions as indicating that rational believers are free to 
decide to believe or disbelieve a proposition according 
to their preferences? Are they free to adopt the doxastic 
preferences they wish?° 


5. Iterated Changes of Belief 


There is a broad consensus about how to use selection 
structures in one-shot belief revision: take the AGM solu- 
tion. However, iterated belief change or, equivalently, 
the change of selection structures exhibits substantial 
ambiguities (Darwiche & Pearl, 1997). Even within the 
narrow confines of simple qualitative theories (no num- 
bers, only propositional inputs), there are quite a few 
models that have been proposed. They typically adopt 
the AGM postulates for one-shot belief revision and add 
one rationality postulate taking care of iterations. We 
look at five important proposals. 

The first model uses one and the same selection struc- 
ture for all possible belief sets. This structure does not rep- 
resent the reasoner’s current belief state but is external to it 
(Areces & Becher, 2001). In contrast, the other four mod- 
els identify the reasoner’s belief state with her selection 
structure. The second model is maximally conservative: the 
selection structure is changed so that the input just gets 
accepted, but the changes made are absolutely minimal, 
and as a consequence, the newly acquired information is 
very easily lost in further belief changes (Boutilier, 1996). 
The third model, restrained revision, is close to the con- 
servative one but gives the input somewhat more impact 
(Booth & Meyer, 2006). The fourth model is moderate in 
that it recommends giving high priority to the incoming 
information, but it does not wipe out all plausibility dis- 
tinctions between worlds or situations that do not com- 
ply with it (Nayak, 1994). The fifth model is radical in 
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that the input is given uncompromising priority over the 
previous beliefs: any world or situation that does not sat- 
isfy the input is considered maximally implausible after 
the revision (Segerberg, 1998). Each of these models is 
well motivated by a possible-worlds semantics and can be 
axiomatically characterized by a single axiom taking care 
of the iteration case: it equates (K*A)*B with K*(A a B), 
if B is consistent with K*A (external and conservative 
revision); if B is consistent with K*A and A is consistent 
with KB (restrained revision); if B is consistent with 
A (moderate revision); or in any case (radical revision). 
And it equates (K*A)*B with K*B otherwise (exception: 
with LB for external revision’). The belief set resulting 
from any finite number of revision steps can be gener- 
ated inductively by repeated applications of the respec- 
tive axiom. We can see that according to each model, two 
successive revision steps can be reduced to a one-shot 
revision, mostly by means of some characteristic case dis- 
tinction (but this kind of reducibility has recently been 
challenged by Booth & Chandler, 2017). Each of these 
competing models for iterated belief change is adequate 
for some cases, but it is easy to come up with examples 
showing that they have drawbacks in other cases. Unfor- 
tunately, no methodology has been put forward yet that 
would tell us when it is rational to use which model. 


6. Further Developments 


The AGM model is a classic by now that has in many 
ways been extended and revised. 


6.1 Extensions of Belief Revision 

Multiple revisions are occasioned by sets of sentences 
that have to be accepted or withdrawn simultaneously 
(Delgrande & Jin, 2012; Fuhrmann & Hansson, 1994). 
In nonprioritized belief revision, a reasoner does not invari- 
ably incorporate the new information but first decides 
in the light of her current belief state whether or not 
she wants to revise her beliefs at all (Hansson, 1997). 
Two-dimensional belief revision is a method that changes 
beliefs in response to imperatives of the form “Accept A 
with a degree of plausibility that at least equals that of 
B” (Cantwell, 1997; Fermé & Rott, 2004; Rott, 2012). Put 
differently, this belief change operation takes two argu- 
ments, an input sentence A and a reference sentence B, or 
the input is of the form “B < A.” This method of chang- 
ing belief states can be used for belief revision (with 
respect to the input sentence) as well as for belief con- 
traction (with respect to the reference sentence). Two- 
dimensional belief revision is much more flexible than 
the methods of iterated belief revision mentioned in 
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section 5. For instance, beliefs can be lowered, and non- 
beliefs can be raised, in doxastic status while still remain- 
ing beliefs and nonbeliefs, respectively. 


6.2 Belief Merging 

The field of belief merging or belief fusion is concerned 
with the rational aggregation of two or more belief states 
(Baral, Kraus, Minker, & Subrahmanian, 1992; Konieczny 
& Pino Pérez, 2002; Nayak, 1994; Pigozzi, 2015). In many 
variants of merging, negotiation procedures or aggrega- 
tion processes similar to those known from social choice 
theory find application. While belief merging should be 
thought of as primarily combining belief states of several 
agents, Konieczny and Pino Pérez (2002) also make room 
for propositional “inputs” in the original AGM sense 
(they call them “integrity constraints”). They show that 
the merging postulates can be modified or supplemented 
in a wide variety of interesting ways, provide a construc- 
tive approach, and give appropriate representation theo- 
rems for basic as well as for supplemented models. 


6.3 Updates 

Katsuno and Mendelzon (1992) discovered an important 
distinction between belief revisions and belief updates. 
Revisions are prompted by new information about an 
unchanging world, while updates are prompted by infor- 
mation about changes in the world. The difference is 
most conspicuous in the case where the new piece of 
information A is consistent with the belief set K. Revi- 
sions by A make a person eliminate from the set of worlds 
compatible with K those worlds that do not satisfy A. If 
you learn that Lisa is the owner of the theater, with the 
understanding that she has owned it for some time, then 
you simply discard all possible worlds in which she is not 
the owner (this is in accordance with dynamic coher- 
ence). In contrast, if you learn that Lisa is the owner 
of the theater with the understanding that she has just 
bought it, then you don’t drop any of your possible 
worlds but modify each of them individually by “letting 
Lisa become” the owner of the theater. An update by A 
makes a person consider, for each world compatible with 
K, how this world would develop if A were brought about 
in it. A formal mark of such updates is that they violate 
postulate (*4) and instead satisfy the following monoto- 
nicity condition: 


(«M) If .K; C Ko, then K;*A € K,*A. (*-Monotonicity) 
6.4 Default Inferences as Expectation Revision 


Given the information that Tweety is a bird, you will 
typically conclude that Tweety can fly. But given the 


Belief Revision 


information that Tweety is a penguin, which of course 
implies that Tweety is a bird, you will not infer that 
Tweety can fly. In commonsense reasoning, an exten- 
sion of the set of premises not only gains information 
but often loses some conclusions that were drawn on the 
basis of the smaller premise set. Everyday inferences thus 
do not conform to the rules of classical logic but exhibit 
patterns of defeasible or nonmonotonic reasoning. Most 
conspicuously, they violate the following rule of ordinary 
(Tarskian) logics: 


(M) If HCH", then Inf(H) C Inf(A'). (Unf-Monotonicity) 


Gardenfors and Makinson (1994) suggested that the set 
of defeasible consequences Inf(H) of a finite set of premises 
H = {Aj,..., An} can be regarded as the theory that results 
from the revision of some set of background assumptions, 
defaults, or expectations E by the conjunction of the prem- 
ise set H. They proposed the following equation: 


(1) Inf(H) = E*(Aja...0A,). 


For the inference operation Inf thus defined, adding 
information can defeat previously drawn conclusions, 
just as when we enlarge the singleton premise set con- 
taining only “Tweety is a bird” (Ai) with the premise 
“Tweety is a penguin” (Az). Given our expectations, the 
input A; makes us conclude that Tweety can fly, whereas 
conjoining the input A, will make us draw the opposite 
conclusion (note that, given our background knowl- 
edge, A; A Az is equivalent to A2 alone). 

Defeasible or nonmonotonic inference operations are 
quite irregular as compared to Tarskian logics. The failure 
of (M) is indeed incisive. However, commonsense reason- 
ing nevertheless appears to leave intact quite a number of 
important classical inference patterns (Brewka, Marek, & 
Truszczynski, 2011; Kraus, Lehmann, & Magidor, 1990; 
Makinson, 1994, 2005). Gardenfors and Makinson (1994) 
proved correspondences of the basic and full sets of ratio- 
nality postulates for belief revision and appropriate sets of 
rationality postulates for default reasoning, and numer- 
ous other correspondences are detailed in Rott (2001). 
Results such as these suggest that belief change and defea- 
sible reasoning are indeed just “two sides of the same 
coin” (Gardenfors, 1991). 


7. Conclusion 


Research on rational belief change originally started 
out in philosophy and is now most actively pursued in 
computer science, but it is an essentially interdisciplin- 
ary undertaking. One of the tasks ahead is to develop 
a methodology for when to apply which of the many 
models on offer. Another task is to close the gap between 
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idealized models of perfectly rational reasoners and the 
actual facts about the limited logical and computational 
capacities of human beings. But whether we view it from 
a normative or an empirical perspective, the formation 
and transformation of beliefs is a problem that raises its 
head just about everywhere. 


Notes 


1. Many authors present revision functions as two-place func- 
tions taking a pair of a belief set and a sentence and returning a 
new belief set. This way of putting things appears misleading to 
me, since one and the same belief set may well be coupled with 
IK, A 
cannot be functional in the K argument. For more details, see 
Rott (1999). 


different belief revision strategies, so * as acting on pairs 


2. The consistency requirement can be seen as marking off the 
enterprise of belief revision from that of paraconsistent logic, 
which shares the aim of explicating rational deliberation in the 
face of contradictory information. A logic or inference operation 
is paraconsistent if it does not satisfy the classical rule of ex falso 
quodlibet (i.e., if not everything is derivable from a classically 
inconsistent set of premises). Wassermann (2011), Girard and 
Tanaka (2016), and Testa, Coniglio, and Ribeiro (2017) argue 
that one should combine belief revision and paraconsistent 
logics. 


3. See Rott (2003). The thesis that recipes for belief revision 
ought to be derived from decision-theoretic principles has been 
championed by Isaac Levi at least since Levi (1991). 


4. For more detailed overviews, see Gdardenfors and Rott 
(1995), Hansson (1999), and Fermé and Hansson (2018). 


5. A great variety of representation theorems can be found 
in Gardenfors (1988), Hansson (1999), Bochman (2001), Rott 
(2001), and Fermé and Hansson (2018). 


6. This raises the question of doxastic voluntarism, which is 
linked with belief revision theory in Rott (2017). 


7. “External” methods based on global (i.e., belief-state inde- 
pendent) similarities or distances between possible worlds 
behave differently (cf. Lehmann, Magidor, & Schlechta, 2001). 
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5.3 Ranking Theory 


Gabriele Kern-Isberner, Niels Skovgaard-Olsen, and Wolfgang Spohn 


Summary 


Ranking theory is one of the salient formal representa- 
tions of doxastic states. It differs from others in being 
able to represent both belief in a proposition (i.e., tak- 
ing it to be true) and degrees of belief (i.e., beliefs as 
more or less firm) and thus to generally account for 
the dynamics of these beliefs. It does so on the basis of 
fundamental and compelling rationality postulates and 
is hence one way of explicating the rational structure 
of doxastic states. Thereby, it provides foundations for 
accounts of defeasible or nonmonotonic reasoning. It 
has widespread applications in philosophy, it proves 
to be highly useful in artificial intelligence, and it has 
started to find applications as a model of reasoning in 


psychology. 
1. Belief and Degrees of Belief 


Epistemic or doxastic attitudes’ representing how the 
world is like come in degrees, whether you call them 
degrees of belief, of uncertainty, or of plausibility (there 
are more terms around). There are various accounts of 
those degrees, amply presented in this handbook.” The 
interests in those accounts are manifold: philosophers 
are concerned with the rational nature of those degrees, 
artificial intelligence (AI) researchers are interested in 
their computational feasibility, psychologists deal with 
their actual manifestations, and all sides argue about 
how well they are suited to model human reasoning. 

However, we also have the notion of belief simpliciter. 
Related notions are those of acceptance or judgment. 
These are indeed the more basic notions when it comes 
to truth, to truly representing the world. Beliefs can be 
true, but degrees of belief cannot. The latter rather relate 
to action (see chapter 8.2 by Peterson, this handbook). 
Accounts of degrees of belief invariably have great diffi- 
culties in doing justice to this fundamental point. There 
is a questionable tendency to take degrees of belief as 
basic and to belittle those difficulties. 


So we need to theoretically account for belief simplic- 
iter. The first attempt was doxastic logic (see chapter 5.1 
by van Ditmarsch, this handbook). However, it is static 
and lacks a dynamic perspective. This perspective has 
been unfolded in belief revision theory (see chapter 5.2 
by Rott, this handbook). However, this theory has prob- 
lems with iterated belief revision, which is required for a 
complete dynamic account. 

Ranking theory promises both to represent belief and 
degrees of belief and to provide a complete dynamics 
for both. These features give it a prominent place in the 
spectrum of possible theories. It was first presented in 
English in Spohn (1988) and fully developed in Spohn 
(2012). Easy access is provided in Spohn (2009). Its far- 
reaching applications in philosophy of science, episte- 
mology, and even normative reasoning may be found, 
for example, in Spohn (2012, 2015, 2020). There is no 
place here to go into any of them. 

Below, we present the basics of the theory in section 2 
and its dynamic aspects in section 3. Section 4 is com- 
parative. Section 5 gives a short introduction to the the- 
ory’s relevance for artificial intelligence, and section 6 
explains how it can be put to use in psychology. 


2. The Basics of Ranking Theory 


Grammatically, “believe” is a transitive verb. In the 
phrase “a believes that p,” “a” refers to a (human) sub- 
ject and “that p” seems to be the object. What does “that 
p” stand for? That is, what are the objects of belief? This 
is a difficult and most confusing issue extensively dis- 
cussed in philosophy (under the rubric “propositions”; 
see, e.g., McGrath, 2012). Here, we cut short the issue, 
as is usual in formal epistemology, by saying in a non- 
committal way that “that p” stands for the proposition 
expressed by “p,” where that proposition is its truth con- 
dition, the set of possibilities or possible worlds in which 
p obtains or “p” is true. 

Hence, we simply assume a set W of (mutually exclu- 
sive and jointly exhaustive) possibilities. These may be 
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coarse-grained and refer only to a few things of inter- 
est; they need not consist in entire possible worlds. Each 
subset of W, that is, each element of the power set P(W) 
of W, is a proposition. 

Now, the basic representation of a belief state is sim- 
ply as the set of propositions believed or taken to be 
true in that state, its belief set. Traditionally, a belief set 
BC PW) has to satisfy two rationality requirements: Z 
must be consistent, that is, 1% « @, and # must be deduc- 
tively closed, that is, if CA, then AES. 

These two rationality requirements may seem entirely 
obvious. The rationale of deductive logic is to check 
what we must not believe and what we are committed 
to believe. Note, however, that deductive closure is lost 
when we identify belief with subjective probability above 
a certain threshold: it easily happens that the probabili- 
ties of two propositions are above the threshold, while 
that of their conjunction is below. Thus, the lottery and 
the preface paradox and the general desire to stick to a 
probabilistic representation of belief have led to a con- 
testation of these requirements (see, e.g., Christensen, 
2004). Here we stick to them as absolutely basic (see 
chapter 3.1 by Steinberger, this handbook). Of course, 
these requirements can be maintained only under a dis- 
positional understanding of belief; occurrent thought 
cannot be deductively closed (see also section 6 of the 
introduction, this handbook). 

The notion of a belief set is static. However, belief sets 
continuously change, and we must account for how they 
change (or should rationally change). We cannot do so 
on a qualitative level. In those changes, we often give 
up old beliefs and replace them by new ones, and then 
we give up less well-entrenched beliefs and keep better- 
entrenched ones (see chapter 5.2 by Rott, this hand- 
book). Roughly, this calls for some entrenchment order 
or, indeed, for some kind of degrees of belief measuring 
the strength of entrenchment. Here, ranking theory com- 
mences. Let us start with some brief formal explanations. 


Definition 1: « is a negative ranking function for W iff « 
is a function from A(W) into the set of natural num- 
bers plus infinity o such that for all A, BC W, 


(1) x(W) =O and x(@) =~, 
(2) «(AvB)=min{x(A), «(B)} (the law of disjunction).’ 


The basic interpretation is that « expresses degrees of 
disbelief (whence the qualification “negative”). If «(A) =0, 
A is not disbelieved at all. This allows that «(-A) =0 as 
well (where 7A is the negation of A); in that case, we 
have indifference or suspension of judgment regarding 
A. If k(A)>0, A is disbelieved or taken to be false, and 
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the more so the larger «(A). So, positive belief in A is 
expressed by disbelief in —A, that is, by x(-A) > 0 (which 
implies that «(A) =0). 

This interpretation explains axioms (1) and (2): 
axiom (1) says that the tautology W is not disbelieved, 
and hence the contradiction © is not believed. This 
entails that beliefs are consistent according to x. Axiom 
(1) moreover says that the contradiction is indeed maxi- 
mally disbelieved. And axiom (2) states that you cannot 
disbelieve a disjunction less strongly than its disjuncts. 
This entails in particular that if you believe two con- 
juncts, you also believe their conjunction. Hence, beliefs 
are deductively closed according to x. In other words, 
the belief set & = {A | k(-A) > 0} associated with x satisfies 
the two basic rationality requirements. Note, moreover, 
that (1) and (2) entail 


(3) either «(A) =0 or K(7A)=0 (or both) (the law of 
negation), 


that is, you cannot (dis)believe A and ~A at once. 

For an illustration, consider Tweety. Tweety has, or fails 
to have, each of three properties: being a bird (B), being 
a penguin (P), and being able to fly (F). This opens up 
eight possibilities. Suppose you have no idea who or what 
Tweety is, but somehow you do not think that it might 
be a penguin. Then your negative ranks for the eight pos- 
sibilities (which determine the ranks for all other propo- 
sitions) may be as given in table 5.3.1 (chosen in some 
plausible way—but see below how the numbers may be 
justified). 

In this case, the strongest proposition you believe is 
that Tweety is either no penguin and no bird (~Ba ~P) 
or a flying bird and no penguin (Fa B ~P); all other pos- 
sibilities are disbelieved. Hence, you neither believe that 
Tweety is a bird nor that it is not a bird. You are also 
indifferent concerning its ability to fly. But you believe, 
for example, that if Tweety is a bird, it is not a penguin 
and can fly (-Bv(-PaF)), and that if Tweety is a pen- 
guin, it can fly (-P v F)—each if-then taken as material 
implication. Surely you believe the latter only because 
you believe that Tweety is not a penguin in the first 
place. The large ranks in the last column indicate your 
strong disbelief in penguins not being birds. This may 
suffice as a first illustration. 


Table 5.3.1 
Prior beliefs about Tweety 


K Ba7-P BaP “BaP 


F (0) 4 0) ole 
-F 2 1 0 8 


Ranking Theory 


We will see the reasons for starting with negative 
ranks. But of course, we can also introduce the positive 
counterpart to « by defining f to be a positive ranking 
function iff there is a negative ranking function « such 
that B(A) = «(~A) for all propositions A. The function 6 
represents degrees of belief. Of course, (1) and (2) translate 
directly into axioms for f. 

We may as well represent degrees of belief and degrees 
of disbelief in a single function by defining t to be a two- 
sided ranking function iff there is a negative ranking func- 
tion « and the corresponding positive ranking function f 
such that t(A) = P(A) - K(A) = k(7A) - «(A) for all proposi- 
tions A. Thus, we have 1t(A) > 0, <0, or =0 according to 
whether A is believed, disbelieved, or neither in t. There- 
fore, this is perhaps the most intuitive notion. However, 
the mathematics is best done in terms of negative rank- 
ing functions. It is clear, though, that the three functions 
are interdefinable. 

There is an important interpretational degree of free- 
dom that we have not yet noticed. So far, we said that 
belief in A is represented by «(7A) = B(A) = T(A) > 0. How- 
ever, we may often find it useful to raise the threshold 
for belief, as we do informally in asking, “Do you really 
believe A?” That is, we may as well say that belief in A 
is only represented by «(7A) = f(A) = t(A) > z for some 
z=0. This seems to be a natural move: belief is vague. 
Where does it commence, and when does it cease? And 
this vagueness seems well represented by that parame- 
ter z. This move at the same time enlarges the range of 
suspension of judgment to the interval from -z to z. The 
remarkable point about axioms (1) and (2) is that they 
guarantee belief sets to be consistent and deductively 
closed, however we choose the threshold z. They are 
indeed equivalent to this general guarantee. 


3. Conditional Ranks, Reasons, and the Dynamics 
of Ranks 


So far, we have sketched only the static part of ranking 
theory. However, we mentioned that the numeric ranks 
are essentially used to account for the dynamics of belief; 
they are not just to represent greater and lesser firmness 
of (dis)belief. To achieve this, the crucial notion is that 
of conditional ranks: 


Definition 2: Let « be a negative ranking function for 
W and x(A) < ~. Then the conditional rank of B given A 
is defined as k(B| A) = K(Aa B)- K(A). 


We might rewrite this definition as 


(4) «(An B)=«k(A)+K(B|A) (the law of conjunction). 
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This is highly intuitive. For, what is your degree of dis- 
belief in A ~a B? One way for A a B to be false is that A is 
false; this contributes «(A) to that degree. However, if A 
is true, B must be false, and this adds «(B | A). 

It immediately follows for all propositions A and B 
with «(A) < © that 


(5) «(B|A)=0 or «(-B|A)=0 (the conditional law of 
negation). 


This law says that even conditional belief must be con- 
sistent. If both «(B| A) and x(~B| A) were > 0, both B 
and ~B would be (dis)believed given A, and this must be 
excluded, as long as the condition A itself is considered 
possible. Indeed, given definition 2 and axiom (1), we 
could axiomatize ranking theory also by (5) instead of 
(2). Hence, the only substantial assumption written into 
ranking functions is conditional consistency. 

Axioms (1) and (2) did not refer to any cardinal prop- 
erties of ranking functions. However, the definition of 
conditional ranks involves arithmetical operations and 
thus presupposes a cardinal understanding of ranks. We 
will see below how this may be justified. We hasten to 
add that one could as well define positive conditional 
ranks by 6(B|A) =«(-B|A) and two-sided conditional 
ranks by t(B| A) = K(7~B| A) - «(B| A). 

As an illustration, consider again table 5.3.1 and 
the conditional beliefs contained therein. We can see 
that precisely the (material) if-then propositions non- 
vacuously held true correspond to conditional beliefs. 
According to the x« specified, you believe, for example, 
that Tweety can fly given it is a bird (since x(-F |B) = 1), 
and also given it is a bird but not a penguin (since 
«(7F|Ba =P) =2) and that Tweety cannot fly given it is 
a penguin (since «(F | P) = 3). Hence, your vacuous belief 
in the material implication “If Tweety is a penguin, it 
can fly” does not amount to a corresponding condi- 
tional belief. In other words, “if ...then...” expresses 
conditional belief rather than material implication, as 
suggested by the Ramsey test for conditionals (see also 
chapter 6.1 by Starr, this handbook). 

A first fundamental application of conditional ranks 
lies in the notion of an epistemic reason, which is at the 
center of the entire handbook. It is very natural to say 
that A is a reason for B iff A speaks in favor of B or con- 
firms B, if A makes B more plausible or less implausible, or 
if B is more credible or less incredible given A than given 
7A. This explanation works for any conception of condi- 
tional degrees of belief. In a probabilistic interpretation, it 
amounts to Carnap’s notion of incremental confirmation 
or positive relevance (Carnap, 1950/1962), which is basic 
for confirmation theory.* Ranking-theoretically, it leads to 
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Definition 3: A is a reason for B relative to the negative 
ranking function « or the associated two-sided rank- 
ing function tiff 7(B| A) > t(B| 7A). 


We can show that if A is a deductive reason for B, 
that is, if ACB, then A is also a reason for B according to 
definition 3 (given K(A), «(-B) <). Clearly, this defini- 
tion, depending entirely on the subject’s doxastic state, 
provides only a subjectively relativized notion of a rea- 
son. Philosophers strive for a more objective notion of 
a reason, perhaps because they take objective deductive 
reasons as a paradigm. In our view, the extent to which a 
more objective notion may be reached is a philosophically 
fundamental, alas very open, issue (see Spohn, 2018). 

On this account, reasons can take four significant forms, 
depending on whether t(B|A) and t(B| 7A) are positive or 
negative. For example, A is a sufficient reason for B iff B is 
believed given A and not believed given ~A. We suggest 
that this is indeed the core meaning of the term “sufficient 
reason,” although it is often used differently. 

Moreover, we may define that B is (doxastically) irrele- 
vant to or independent of A if neither A nor ~A is a reason 
for B. On this basis, a theory of (conditional) inde- 
pendence can be developed in ranking terms in wide- 
ranging analogy to the probabilistic theory. For instance, 
the theory of Bayes nets as developed by Pearl (1988) 
(see also chapter 4.2 by Hartmann, this handbook) could 
equally well be stated in terms of ranking theory (see 
Spohn, 1988; Goldszmidt & Pearl, 1996). 

A second fundamental application of conditional 
ranks lies in the dynamics of beliefs and ranks. As in prob- 
ability theory, we may say that, upon learning evidence 
E, we should simply move to the degrees of belief con- 
ditional on E. Thereby, though, the evidence E acquires 
maximal certainty, that is, probability 1 or positive rank 
co, This seems too restrictive. In general, evidence may 
be (slightly) uncertain, and our rules for doxastic change 
through evidence or learning—we do not attend to 
changes caused in other ways, like forgetting—should 
take account of this. In ranking theory, this is achieved 
by two principles: first, conditional ranks given the evi- 
dence E and given its negation ~E are not changed by 
the evidence itself—how could it change them?—and 
second, the evidence E does not become maximally cer- 
tain but improves its position by n ranks, where n is a 
free parameter characterizing the specific information 
process.° These two assumptions suffice to uniquely 
determine the kinematics of ranking functions (i.e., 
ranking-theoretic conditionalization). 

In order to see how this works, look again at our 
Tweety example: suppose you learn in some way, and 
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accept with firmness 2, that Tweety is a bird. Thus, you 
shift up ~B-possibilities by 2 and keep constant the rank 
differences within B and within ~B. This results in the 
posterior ranking function «' given in table 5.3.2. 

In x’, you believe that Tweety is a bird able to fly but 
not a penguin; you still neglect this possibility. So, in x’, 
you believe more than in x; in belief revision theory (cf. 
chapter 5.2 by Rott, this handbook), this would be called 
a “belief expansion.” 

Next, to your surprise, you tentatively learn and 
accept, say, with firmness 1 that Tweety is indeed a pen- 
guin. This results in another ranking function «”, which 
shifts all P-possibilities down by 1 and all ~P-possibilities 
up by 1, so that P is indeed believed with firmness 1 (i.e., 
«''(=P) = 1; see table 5.3.3). 

So, you have changed your mind and believe in x” 
that Tweety is a penguin bird that cannot fly. In belief 
revision theory, this would be called a “belief revision.” 
Obviously, belief contraction (cf. chapter 5.2 by Rott, 
this handbook), where you simply give up a belief pre- 
viously held without replacing it by a new one, can 
also be modeled by ranking-theoretic conditionaliza- 
tion. The example already demonstrates that this rule 
of belief change can be iteratively applied ad libitum. 

An important application of ranking-theoretic condi- 
tionalization is that it delivers a measurement procedure 
for ranks that justifies the cardinality of ranks. This proce- 
dure refers to iterated belief contraction. Its point is this: 
if your iterated contractions behave as prescribed by rank- 
ing theory,’ then this behavior uniquely determines your 
ranking function up to a multiplicative constant. That is, 
your ranks can thereby be measured on a ratio scale (see 
Hild & Spohn, 2008). The consequences of the fact that 
ranks are measured only on a ratio scale await investiga- 
tion. They imply, for example, a problem analogous to 
the problem of the interpersonal comparison of utilities. 


Table 5.3.2 
Learning that Tweety is a bird 


x’ Ba7P BaP -“Ba7P “BaP 
F 0) 4 2 13 

“F 2 1 2 10 
Table 5.3.3 

Learning that Tweety is a penguin 

K" Ba7-P BaP “BaP “BaP 
F 1 3 12 
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4. Comparisons 


The formal structure defined by axioms (1) and (2) has 
been called Baconian probability by Cohen (1980). Its first 
clear appearance is in the functions of potential surprise 
developed by Shackle (1949). The structure is also hid- 
den in Rescher (1964) and is clearly found in Cohen’s 
own work (1970, 1977). The crucial formal advance 
of ranking theory lies in the definition of conditional 
ranks, which cannot be found in these works and makes 
the theory a properly cardinal one. 

Belief revision theory was precisely about the dynam- 
ics of belief. However, it only conceived of entrench- 
ment orders. And within their ordinal framework, there 
was no Clear solution of the problem of iterated belief 
revision (see chapter 5.2 by Rott, this handbook). 

Possibility theory, building on early work by Zadeh 
(1978) and developed by Dubois and Prade (see chap- 
ter 4.7 by Dubois & Prade, this handbook), is in fact 
equivalent to ranking theory; axiom (2) is the charac- 
teristic property of possibility measures. However, the 
interpretation of those measures was intentionally left 
open, leaving considerable formal uncertainty as to how 
to conceive of conditional degrees of possibility. 

The theory of Dempster-Shafer (DS) belief functions, 
as developed in Shafer (1976), seems to be a far more 
general theory (see again chapter 4.7 by Dubois & Prade, 
this handbook), which comprises probability theory 
and also ranking theory as a special case. Shafer (1976, 
chapter 10) defines so-called consonant belief func- 
tions, which appear to be equivalent to negative ranking 
functions. However, their respective dynamic behaviors 
diverge, a fact that prevents reduction of ranking theory 
to DS theory (see Spohn, 2012, section 11.9). 

Of course, the largest comparative issue is how rank- 
ing theory relates to probability theory. A comparison of 
their axioms and their form of conditionalization suggests 
translating the sum of probabilities into the minimum of 
ranks and the product and the quotient of probabilities 
into the addition and the subtraction of ranks, respec- 
tively. This translation works only for negative ranks; that’s 
why they provide the formally preferred version of rank- 
ing theory. And it explains why very many things that 
can be done with probability theory also work for ranking 
theory in a meaningful way. 

However, the translation does not justify conceiving 
ranks in probabilistic terms. As mentioned in section 2, 
belief in A cannot be probabilistically represented by 
P(A) =1-<€ if one sticks to the consistency and deduc- 
tive closure of belief sets. The relation of probability and 
belief is hotly debated in philosophy, without a clear 
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solution emerging (see, e.g., the proposals of Leitgeb, 
2017, and Raidl & Skovgaard-Olsen, 2017). Therefore, 
our attitude has always been to independently develop 
ranking theory as a theory of belief. 


5. Ranking Functions in Artificial Intelligence 


Besides probability theory and logic, ranking functions 
are among the most popular formalisms used for knowl- 
edge representation® and reasoning (KRR), and their 
popularity is still increasing because they provide a very 
versatile framework for many central operations in KRR, 
as already sections 2 to 4 pointed out. Most importantly, 
ranking functions are a convenient common basic tool 
for nonmonotonic reasoning and belief revision. Belief 
revision already has been explained in more detail in 
section 3, and nonmonotonic reasoning also deals with 
belief dynamics insofar as conclusions may be given up 
when new information arrives (so, the consequence rela- 
tion is not monotonic, as in classical logic). Both fields 
emerged in the 1980s (partly) as a reaction to the inca- 
pability of classical logic to handle problems in everyday 
life that intelligent systems like robots were expected to 
tackle. Knowledge or belief about the world is usually 
uncertain, and the world is always changing. Therefore, 
AI systems built upon classical logics failed. So-called pref- 
erential models (Makinson, 1989) provide an important 
semantics for nonmonotonic logics. Their basic idea is to 
order worlds according to normality and to focus on the 
minimal ones (i.e., the most plausible ones) for reasoning. 
Likewise, AGM belief revision theory (see chapter 5.2 by 
Rott, this handbook) needs orderings of worlds to become 
effective. For both fields, ranking functions offer a quite 
perfect technical tool that also complies nicely with the 
intuitions behind the techniques. Moreover, they can 
also evaluate conditionals and are an attractive qualita- 
tive counterpart to probabilities (see section 3). 

Judea Pearl was probably the first renowned AI scien- 
tist to make use of ranking functions; his famous system 
Z (Pearl, 1990) is based on them. He has continuously 
emphasized the commonsense structural qualities of 
probabilities and developed ranking functions as an 
interesting qualitative counterpart to probabilities. He 
set up his system Z as an “ultimate system of nonmono- 
tonic reasoning” in terms of ranking functions. To date, 
it is one of the best and most convenient approaches to 
implement high-quality nonmonotonic reasoning. 

Consequently, ranking functions are deeply con- 
nected with nonmonotonic and uncertain reasoning 
and with belief change, which are core topics in KRR. 
Many researchers make use of them in one way or 
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another even if they rely on more general frameworks. 
Darwiche and Pearl (1997) presented general postulates 
for the iterated revision of general epistemic states but 
illustrated their account with ranking functions. So did 
Jin and Thielscher (2007) and Delgrande and Jin (2012) 
when they devised novel postulates for iterated and mul- 
tiple revision. Interestingly, the independence properties 
for advanced belief revision that were proposed in those 
studies can be related to independence with respect to 
ranking functions (see Spohn, 2012, chapter 7) in anal- 
ogy with probabilistic independence (see Kern-Isberner 
& Huvermann, 2017). 

Indeed, as suggested in section 3, ranking functions 
are particularly well suited for iterated belief change 
because they can easily be modified in accordance with 
AGM theory, returning new ranking functions that are 
readily available for a subsequent change operation. The 
main AGM operations are revision (adopting a belief) 
and contraction (giving up a belief), related by Levi and 
Harper identities (see chapter 5.2 by Rott, this handbook). 
In ranking theory, the connections between these opera- 
tions are even deeper, since (iterated) contraction is just 
a special kind of (iterated) ranking conditionalization. 
Indeed, the results of Kern-Isberner, Bock, Sauerwald, 
and Beierle (2017) show that iterated revision and con- 
traction can be performed by a common methodology. 

Continuing on that, and beyond the practicality and 
diversity of ranking functions, it is crucial to under- 
stand that they are not just a pragmatically good choice 
but indeed allow for deep theoretical foundations of 
approaches to reasoning. It is the ease and naturalness 
with which they can handle conditionals—very similar 
to probabilities—that make them an excellent formal 
tool for modeling reasoning. Given that conditionals 
are, on the one hand, crucial entities for nonmonotonic 
and commonsense reasoning and belief change, and, on 
the other hand, formal entities fully accessible to con- 
ditional logics, this capability provides a key feature for 
logic-based approaches connecting nonmonotonic log- 
ics and belief change theories with commonsense and 
general human reasoning. More precisely, conditional 
ranks give meaning to differences between degrees in 
belief when observing A versus A a B (see the law of con- 
junction in section 3), and the examples of belief change 
given in section 3 illustrate nicely how it is easily pos- 
sible to preserve these differences under change when 
using ranking functions. This property has been elab- 
orated as a principle of conditional preservation in Kern- 
Isberner (2004), giving rise to defining c-representations 
and c-revisions (all belief changes shown in section 3 are 
c-revisions). C-representations are c-revisions starting 
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from a uniform ranking function and allowing for rea- 
soning from conditional belief bases. Ranking theory is 
one of the few formal frameworks (probability theory 
is another) that is rich and expressive enough to allow 
such a precise formalization of conditional preservation 
that supports both belief change and inductive reason- 
ing as a common methodology. 


6. Ranking Theory in Psychology 


Potentially, ranking theory has applications in many 
areas of psychology. To illustrate, psychological research 
on belief revision has been carried out under the inspira- 
tion of AGM theory and probabilistic updating (Barat- 
gin & Politzer, 2010; Oaksford & Chater, 2013; Wolf, 
Rieger, & Knauff, 2012; see also chapter 5.4 by Gazzo 
Castafieda & Knauff, this handbook). Such work could be 
extended by ranking theory, given that one of its central 
motivations was to represent a notion of full belief, in con- 
trast to probabilistic update mechanisms, while improving 
upon AGM theory to allow for iterative revisions. However, 
as Colombo, Elkin, and Hartmann (2018) note, probabi- 
listic Bayesian approaches are currently enjoying a boom 
in cognitive science to the neglect of alternative formal 
frameworks, like ranking theory. 

As said, possibility theory is mathematically equiva- 
lent to ranking theory, yet differs fundamentally in its 
intended interpretation. In Da Silva Neves, Bonnefon, 
and Raufaste (2002) and Benferhat, Bonnefon, and Da 
Silva Neves (2005), possibility theory was subjected 
to empirical testing. In Da Silva Neves et al. (2002), a 
direct route was chosen by testing whether participants’ 
possibility judgments satisfy the rationality postulates 
codified in System P (see chapter 4.4 by Pfeifer, this hand- 
book) augmented by rational monotonicity (A |» C and not 
A } 7B; therefore AaB } C),’ in case their responses 
violated monotonicity (A |» C; therefore A a B }- C). Inter- 
estingly, no such direct test of ranking theory based on 
the participants’ judgments of disbelief or implausibility 
has yet been made. 

What exists is the following: Isberner and Kern- 
Isberner (2017) investigated whether belief revision with 
ranking functions could retrodict findings of temporal 
delay when processing implausible information in tasks 
where plausibility judgments would interfere with the 
task constraints (a finding known as the “epistemic Stroop 
effect”). A guiding assumption underlying this work is 
that ranking theory can be used to represent the situa- 
tional model that participants construct during language 
comprehension. Moreover, Eichhorn, Kern-Isberner, and 
Ragni (2018) proposed a conditional-logical model based 


Ranking Theory 


on ranking functions that allows for the elaboration of 
plausible background knowledge. 

In Ragni, Eichhorn, Bock, Kern-Isberner, and Tse 
(2017), it was investigated whether ranking theory could 
retrodict the suppression effect (Byrne, 1989), where 
endorsement rates of classically valid modus ponens 
(MP) (A—B,A; therefore B) and modus tollens (MT) 
(A—B,-B; therefore =A) are decreased when further 
premises indicating possible defeaters are presented. To 
illustrate: Normally, inferring “Lisa will study late in the 
library” from the premises “Lisa has an essay to write” 
and “If Lisa has an essay to write, then Lisa will study 
late in the library” would be seen as unproblematic. But 
if the additional premise “If the library is open, then Lisa 
will study late in the library” becomes available, then 
participants are much more reluctant to draw this infer- 
ence. However, as Ragni et al. (2017) show, the infer- 
ence mechanism exploiting ranking functions does not 
in itself retrodict the suppression effect. To do so, fur- 
ther assumptions about the underlying knowledge base 
instantiated in long-term memory need to be made. In 
addition, Ragni et al. (2017) present experiments that 
test ranking theory’s ability to predict the participants’ 
reasoning with MP and MT once keywords indicating 
nonmonotonicity such as “normally” are inserted. Char- 
acteristic of this line of research is that c-representations 
are used to inductively infer ranking functions that sat- 
isfy the constraints set by an assumed knowledge base. 

Moreover, the account of conditionals in Spohn (2013, 
2015) has inspired a series of experiments. Spohn (2013, 
2015) outlines a number of expressive roles of condition- 
als that go beyond the Ramsey test, which merely takes 
conditionals to express conditional beliefs. For instance, 
conditionals may express reason relations as specified in 
definition 3. In Skovgaard-Olsen (2014), a logistic regres- 
sion model was presented both to formulate predictions 
for the participants’ evaluations of the conclusions of 
MP, MT, AC (affirming the consequent: A —B, B; there- 
fore A), and DA (denying the antecedent: A—B, 7A; 
therefore ~B) inferences and to suggest a solution to 
the problem of updating based on conditional informa- 
tion. In Skovgaard-Olsen, Singmann, and Klauer (2016), a 
reason-relation reading of the conditional was contrasted 
experimentally with the Ramsey test in participants’ prob- 
ability and acceptability evaluations of indicative condi- 
tionals, and support was obtained for the reason-relation 
reading. In Skovgaard-Olsen, Kellen, Hahn, and Klauer 
(2019), it was found that there are patterns of individual 
variation in these results. In Skovgaard-Olsen, Singmann, 
and Klauer (2017), participants’ perceived relevance and 
reason-relation evaluations were investigated, and some 
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first evidence for the explications of reason relations 
and perceived relevance introduced above was obtained. 
However, characteristic of this latter line of research is 
that it adopts an indirect route to testing ideas from rank- 
ing theory by exploiting its extension to conditionals in 
Spohn (2013, 2015) and its parallels to probability theory 
(see also Henrion, Provan, Del Favero, & Sanders, 1994). 

Presently, paradigms operationalizing degrees of beliefs 
as probabilities are much better established than tasks 
using ranking functions. This is so despite the fact that the 
arithmetical operations of ranking theory require much 
less computational effort than those of probability theory. 
For instance, it is well known that participants exhibit diffi- 
culties in properly integrating information about base rates 
of the rarity of a given disease in evaluating how likely it 
is that a person has the disease given a positive test result. 
Yet, interestingly, Juslin, Nilsson, Winman, and Lindskog 
(2011) find that the notorious base-rate neglect could be 
reduced when participants are given the tasks in a logarith- 
mic format. Juslin et al. (2011) therefore conjecture that a 
linear, additive integration of information becomes more 
intuitive in the absence of access to overriding analytic 
rules that use a multiplicative format, like probabilities. 

Nevertheless, there is a lack of direct experimental 
investigations of ranking functions. This is perhaps due 
to the following challenges: 


(i) Negative ranking functions are useful for conduct- 
ing proofs. But it initially presents a conceptual chal- 
lenge to think in terms of disbelief in negations of 
propositions as a way of representing full beliefs. 
Two-sided ranking functions solve this problem. But 
they come at the cost of having different rules apply- 
ing to the negative and positive ranges of the scale. 

(ii) An operationalization of ranking functions in 
terms of iterated contractions exists (see above), 
but it has not received the same kind of experi- 
mental implementation as the operationalization 
of probabilities in terms of betting quotients. 


(iii) Since negative ranks take natural numbers from 0 
to infinity, there is no natural way of nonarbitrarily 
dividing the scale into regions of ascending degrees 
of disbelief other than a region of zero disbelief, 
a region of above-zero disbelief, and a region of 
maximal disbelief. In contrast, the crude division 
of the probability scale of real numbers between 
0 and 1 into decimal regions enables participants 
and experimenters to make qualitative differentia- 
tions between low degrees of belief (e.g., [0.0, 0.3)), 
medium degrees of belief (e.g., [0.3,0.7]), and 
strong degrees of belief (e.g., (0.7, 1.0]). 
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If challenges such as these can be overcome in future 
work, ranking theory potentially has a lot to offer to psy- 
chology and to cognitive science more generally. 


Notes 


1. Strictly speaking, “epistemic” only refers to knowledge, 
although it is often used more widely. Because we will talk only 
about belief, we prefer to use “doxastic” throughout. See also 
chapter 5.1 by van Ditmarsch (this handbook). 


2. See chapter 4.1 by Hajek and Staffel, chapter 4.5 by Chater 
and Oaksford, chapter 4.7 by Dubois and Prade, chapter 8.3 by 
Glockner, and chapter 8.4 by Hill (all in this handbook). 


3. We use the notation of propositional logic, not the set- 
theoretic one. No confusion can arise thereby. 


4. See chapter 4.3 by Merin (this handbook). Sections 1-4 of 
that chapter work just as well with ranking theory. 


5. See also chapter 2.1 by Broome, chapter 2.2 by Wedgwood, 
and chapter 12.2 by Smith (all in this handbook). 


6. This is completely analogous to Jeffrey conditionalization in 
probability theory (see Jeffrey, 1983, chapter 11; chapter 4.1 by 
Hajek & Staffel, this handbook). Shenoy (1991) has contributed 
an important variant. 


7. In fact, we need no more than twofold nonvacuous con- 
tractions. 


8. In AI, the distinction between knowledge and belief is usu- 
ally quite vague. 


9. Here, |» represents nonmonotonic or default entailment. 
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5.4 Defeasible Reasoning and Belief Revision in Psychology 


Lupita Estefania Gazzo Castafeda and Markus Knauff 


Summary 


How do people change their mind in the light of new 
information? In defeasible reasoning, people withdraw 
previously drawn inferences (conclusions) in light of new 
evidence. In belief revision, people abandon prior beliefs 
(premises) to accommodate new information. Cognitive 
research aims at developing descriptive theories of these 
cognitive functions, without making strong normative 
claims. However, such normative claims are more or less 
implicit in many cognitive theories of human reasoning. 
This chapter summarizes the experimental state of the 
art in the field and then presents the most important 
psychological theories of defeasible reasoning and belief 
revision. We also discuss whether both cognitive pro- 
cesses are really so different from a cognitive (descrip- 
tive) point of view. Both cognitive processes seem to rely 
on the availability of disablers, the facts that prevent 
people from believing the conclusions or premises of a 
reasoning problem. 


1. Defeasible Reasoning and Belief Revision 
in Human Reasoning 


A fundamental prerequisite of rationality is that humans 
are willing to change their mind in light of new evi- 
dence. For instance, suppose that Jack has learned that 
if he goes to bed late, then he will be tired the next day. 
So, if he is going to bed late, he can conclude that he will 
be tired the next day. But if he finds out that the next 
day is a holiday, then it makes sense for Jack to change 
this conclusion. He can conclude that he will not be 
tired because he is able to sleep in. That is, new informa- 
tion can make people reject previous conclusions that 
would have been drawn otherwise. Sometimes, however, 
the information we get is more than an exception to a 
rule. New information can also conflict with our beliefs. 
Imagine, for instance, that Jack usually has problems 
falling asleep. So the psychotherapist tells him that if he 
meditates in the evening, he will fall asleep more easily. 


Following the advice, Jack meditates in the evening. 
But he is still not able to fall asleep. This is not what he 
expected. Therefore, Jack has to revise his prior beliefs 
and conclude either that the psychotherapist was wrong 
or that he meditated inappropriately. 

These two cases of reasoning are called defeasible rea- 
soning and belief revision, respectively. In defeasible rea- 
soning, people withdraw previously drawn conclusions 
in light of new evidence. Jack withdraws the conclusion 
of being tired after going to bed late because he knows 
that he is able to sleep in. This does not mean that he 
no longer believes that going late to bed makes him 
tired. In fact, in defeasible reasoning, one still believes 
in the prior propositions (basic beliefs), but one knows 
that in light of specific circumstances, some conclu- 
sions (derived beliefs) are not suitable anymore. This is 
different in belief revision, in which new information is 
in conflict with one’s prior beliefs. The fact that Jack is 
not able to fall asleep is inconsistent with what the psy- 
chotherapist told him. This inconsistency between new 
evidence and prior beliefs can only be solved by revising 
these prior beliefs. In other words, in belief revision, it 
is not the conclusion that is changed; rather, one has 
to change prior beliefs in order to reach consistency. In 
the psychology of reasoning, these prior beliefs are often 
given by a set of premises. The conclusion is usually the 
belief that follows from these premises. 

From a formal point of view, defeasible reasoning 
and belief revision are nonmonotonic, that is, conclusions 
can be retracted based on further evidence. Psychologi- 
cally, that means that a person abandons accepted con- 
sequences from previous beliefs (Elio & Pelletier, 1997). 
The investigation of nonmonotonicity has its origins in 
philosophy and artificial intelligence (AI), in which many 
normative theories have been developed to describe how 
intelligent systems can rationally operate with incom- 
plete information in changing worlds (for an overview, 
see Antoniou, 1997). In psychology, however, the inves- 
tigation of defeasible reasoning and belief revision is still 
young. For a long time, psychologists concentrated only 
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on deduction, which is monotonic, that is, a conclusion 
has to follow necessarily from its premises and cannot be 
altered by additional information. However, as our initial 
examples show, it is also sensible to change conclusions 
in light of new evidence. Because of that, psychologists 
have also started to develop several descriptive theories of 
human nonmonotonic reasoning and the psychological 
processes behind defeasible reasoning and belief revi- 
sion. The aim of this chapter is to give an overview of 
this psychological research. 


2. The Empirical Investigation of Defeasible 
Reasoning 


Defeasibility is a central property of human reasoning. 
In everyday life, people are often forced to reason with 
uncertain and incomplete information and thus must 
often change previously drawn conclusions in light of 
new evidence. Most of the psychological research on 
defeasible reasoning focused on conditional inferences and 
the circumstances under which the conclusions from such 
inferences are withdrawn. There are two main experimen- 
tal paradigms: the overt and the covert paradigm (Politzer 
& Bonnefon, 2006). 


2.1 Defeasibility in the Overt Paradigm 

The overt paradigm was introduced by Rumain, Connell, 
and Braine (1983) but made popular by Byrne (1989). 
There are usually two experimental conditions. In the 
simple condition, participants have to solve inferences 
consisting of a conditional premise of the form “Tf p, then 
q,” a categorical premise containing p, q, or their negations 
and a conclusion that has to be evaluated. An example: 


If Jack goes to bed late, then Jack will be tired. 
Jack goes to bed late. 
Jack will be tired. 


In this case, the conclusion is true because this is the 
valid modus ponens (MP) inference: if p (the anteced- 
ent) is true, then qg (the consequent) is also true. Another 
valid inference is modus tollens (MT: if p then g; not-q; 
therefore, not-p). By contrast, the other two inferences, 
affirmation of the consequent (AC: if p then q; q; therefore, 
p) and denial of the antecedent (DA: if p then q; not-p; 
therefore, not-q), are fallacies in classical logic because 
p is only sufficient, but not necessary, for q. Participants 
typically have no problem in recognizing MP inferences 
as logically valid, and they do it also for MT inferences, 
although to a lesser extent. The results regarding AC 
and DA are mixed and highly dependent on the task 
demands (Evans, Newstead, & Byrne, 1993). 
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In the extended condition, though, an additional 
premise is added to the inference. This additional prem- 
ise can contain either a disabler or an alternative (Cum- 
mins, 1995; Cummins, Lubart, Alksnis, & Rist, 1991). 
Disablers are additional preconditions that prevent p 
from leading to q. They are thus relevant for MP and MT 
and can be presented either as concrete facts (e.g., Jack is 
able to sleep in) or as conditionals. An example: 


If Jack goes to bed late, then he will be tired. 

If Jack is able to sleep in, then he will not be tired. 
Jack goes to bed late. 

Jack will be tired. 


Alternatives, by contrast, are alternative reasons for q that 
are not p. They are relevant for AC and DA inferences 
because they make it clear that p is not necessary for q. 
Alternatives can be also phrased either as facts (e.g., Jack 
has a disrupted sleep) or as conditionals. For example: 


If Jack goes to bed late, then he will be tired. 
If Jack has a disrupted sleep, then he will be tired. 
Jack does not go to bed late. 


Jack will not be tired. 


The defeasibility of human reasoning is shown by 
comparing the acceptance rates for conclusions in the 
simple and the extended condition. This comparison 
usually shows that disablers and alternatives affect infer- 
ences differently. When disablers are presented, partici- 
pants accept fewer MP and MT conclusions than without 
disablers (e.g., Byrne, 1989; Byrne, Espino, & Santama- 
ria, 1999; De Neys, Schaeken, & d’Ydewalle, 2003b; Ste- 
venson & Over, 1995). When alternatives are presented, 
participants accept fewer AC and DA conclusions than 
without alternatives (e.g., Byrne, 1989; De Neys et al., 
2003b; Rumain et al., 1983). 


2.2 Defeasibility in the Covert Paradigm 

The covert paradigm was used for the first time by Cum- 
mins (Cummins et al., 1991). In the covert paradigm, 
alternatives and disablers are not presented explicitly as 
additional premises but are just implicitly present in the 
premises. For this, a first group of participants is con- 
fronted with a set of conditionals and asked to gener- 
ate as many disablers and alternatives as they can. For 
example: 


If Jack goes to bed late, then he will be tired. 

Jack goes to bed late, but is not tired (or: Jack is tired, but 
he did not go to bed late). 

Please write down as many factors as you can that could 
make this situation possible. 
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Depending on how many disablers and alternatives par- 
ticipants generate, the conditionals are subdivided in 
those having (1) many disablers and many alternatives, 
(2) many disablers and few alternatives, (3) few disablers 
and many alternatives, and (4) few disablers and few 
alternatives. These conditionals are then embedded in 
MP, MT, AC, and DA inferences and presented to a new 
group of participants who have to indicate how sure 
they are that the conclusion can be drawn. The second 
group of participants never sees the alternatives and dis- 
ablers. Nevertheless, the acceptability of the conclusion 
is a function of the number of disablers and alternatives. 
Participants accept fewer MP and MT conclusions when 
conditionals have many instead of few disablers, and 
accept fewer AC and DA conclusions when conditionals 
have many instead of few alternatives (e.g., Cummins, 
1995; De Neys, Schaeken, & d’Ydewalle, 2003a, 2003b; 
Gazzo Castafieda & Knauff, 2018). 


2.3 The Importance of Content and Knowledge 

One of the main reasons why human reasoning is 
defeasible stems from people’s background knowledge 
and their prior beliefs about the content of condition- 
als (De Neys et al., 2003a, 2003b; Dieussaert, De Neys, 
& Schaeken, 2005; Evans & Over, 2004; Johnson-Laird 
& Byrne, 2002; Oaksford & Chater, 2007). Indeed, the 
relation between knowledge, beliefs, and content is not 
trivial (see chapter 13.3 by Knauff, this handbook). How- 
ever, here we simply use the term “knowledge” in the 
sense of true or justified belief. 

As shown by the findings with the covert paradigm, 
defeasibility does not require an explicit presentation of 
new evidence; instead, people activate their beliefs about 
the content of conditionals even if no explicit disablers or 
alternatives are presented (De Neys et al., 2003b; Vadebon- 
coeur & Markovits, 1999). Therefore, prior beliefs are an 
important prerequisite for defeasible reasoning. With- 
out background knowledge (in the sense of true beliefs), 
people cannot know which information is a disabler or 
an alternative that can affect inferences. Experimentally, 
this has been shown in studies comparing participants’ 
acceptance of conclusions for problems with unfamiliar 
and familiar content (Cummins, 1995; Gazzo Castafieda 
& knauff, 2020; Markovits, 1986), by comparing the 
weighting of domain-specific disablers by laypeople and 
experts (Gazzo Castafieda & Knauff, 2016), and recently 
by varying the familiarity of the person or object in a 
conditional (Gazzo Castafieda & Knauff, 2019). In all 
these studies, the results are consistent: when partici- 
pants do not have domain knowledge, they also make 
more MP, MT, AC, and/or DA inferences. In other words, 
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they do not know the circumstances that could defeat 
the conclusions. It is therefore argued that the retrieval 
of disablers and alternatives affects the perceived suffi- 
ciency and necessity relations between p and q: disablers 
make the antecedent less sufficient for the consequent 
so that fewer MP and MT conclusions are accepted, and 
alternatives make the antecedent less necessary for the 
consequent so that fewer AC and DA conclusions are 
accepted (Cummins, 1995; Thompson, 1994, 1995). 

Another factor related to background knowledge that 
is important for defeasible reasoning is the associative 
strength between the premises and the disablers in mem- 
ory. Some disablers are more strongly associated with 
one’s prior knowledge of how to prevent q from happen- 
ing, and some alternatives are more strongly associated 
with one’s prior knowledge of how to cause q (De Neys 
et al., 2003a; Quinn & Markovits, 1998). For instance, 
in our initial example “If Jack goes to bed late, then he 
will be tired,” the disabler “Jack is able to sleep in” might 
be a stronger disabler than “Jack takes a cold morning 
shower.” Therefore, both the number and the associa- 
tive strength of disablers and alternatives affect people’s 
acceptance of conclusions. The more of these strongly 
associated disablers or alternatives exist, the more prob- 
able it is that at least one will be retrieved, and the more 
probable it is that a conclusion will be withdrawn (De 
Neys et al., 2003a, 2003b; Vadeboncoeur & Markovits, 
1999). Initially, an “all-or-nothing phenomenon” was 
assumed (De Neys et al., 2003b, p. 582): as soon as a 
person retrieves one disabling condition or alternative, 
the corresponding conclusion is rejected (cf. Vadebon- 
coeur & Markovits, 1999). However, in a series of experi- 
ments, De Neys and colleagues (2003b) showed that the 
consideration of disablers and alternatives is gradual: 
every additional disabler or alternative has an impact on 
the degree to which people accept a conclusion. There- 
fore, people’s acceptance of the conclusion is related in 
a monotonic fashion to the number of known disablers 
and alternatives. 

Besides knowledge about the number and strength of 
disablers and alternatives, also their frequency of occur- 
rence is important. Geiger and Oberauer (2007)' noticed 
that it is important to distinguish between the number 
of different disablers and the frequency with which they 
are thought to occur (i.e., p & not-q cases). Sometimes, 
there are many disablers for one conditional, but their 
frequency of occurrence is low (and vice versa). For 
instance, for the conditional “If you open the fridge, 
then the light inside goes on,” there are many disablers 
(e.g., broken fridge, broken light bulb, no electricity), but 
their overall occurrence is low (usually, the light goes 
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on when the fridge is opened). When both factors are 
disentangled, the frequency of exceptions seems to pre- 
dict people’s withdrawal of conclusions more accurately 
than the number of disablers (Geiger & Oberauer, 2007). 
Defeasibility in human reasoning is thus not a phenom- 
enon in which a conclusion is either accepted or rejected 
categorically. Instead, the withdrawal of conclusions hap- 
pens by degrees and depends on how much one knows 
about the circumstances that may affect the conditional 
relationship. 


2.4 The Importance of the Experimental Context 

How the context influences defeasible reasoning has not 
received as much attention as content-related factors. 
However, contextual factors are nonetheless interest- 
ing as they can moderate the degree to which partici- 
pants consider potential disablers and alternatives. For 
instance, Vadeboncoeur and Markovits (1999) showed 
that people’s automatic activation of background knowl- 
edge about disablers can be inhibited by instructions 
emphasizing the logical necessity of conditional infer- 
ences. Furthermore, Markovits, Lortie Forgues, and Bru- 
net (2010) showed that people’s tendency to consider 
the frequency of exceptions can be inhibited by the 
presentation of a dichotomous response format. They 
showed that only when scaled response formats were 
available did information about the frequency of excep- 
tions affect the acceptance of conclusions. But when 
there was only a dichotomous response format, then 
people rejected the conclusion already in light of a sin- 
gle disabler. Finally, the consideration of disablers and 
alternatives can also be moderated by the introduction 
of words questioning the believability of the conditional 
or the disabler. George (1997), for instance, conducted 
experiments where conditional statements were either 
presented traditionally as “if p then q” or with an addi- 
tional “very probable” or “not very probable” before the 
conditional. He found that the certainty of the condi- 
tional statement influenced participants’ belief in the 
conclusion. Stevenson and Over (1995) elicited similar 
effects by introducing doubt in the disabler. Participants 
were confronted with conditionals such as “If John goes 
fishing, he will have a fish supper” and disablers such 
as “If John catches a fish, he will have a fish supper.” 
However, when participants were told, for example, that 
John is always lucky, they rejected fewer conclusions. 


3. The Empirical Investigation of Belief Revision 


In defeasible reasoning, people change their conclu- 
sions (derived beliefs) in light of new evidence. In 
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belief revision, people instead have to make changes 
in their system of prior beliefs. This system of beliefs 
can be rather complex and interconnected, as we know 
from research in AI and philosophy. Psychologists are 
usually more pragmatic and often just treat the set of 
premises in an inference problem as the set of prior 
beliefs. Although this is a rather strong simplification, 
it is justified in many experimental settings in the psy- 
chological laboratory. Belief revision is then based on 
the assumption that a set of premises (beliefs) has to be 
consistent. Therefore, in our meditation example, Jack 
has to revise his beliefs: knowing that he did not fall 
asleep, he cannot believe that meditating makes him 
sleep. He has to find a way to reconcile his beliefs with 
what he observed. But which belief will he abandon? 
In AI and philosophy, two main accounts of which 
belief should be rejected have been proposed: mini- 
mal change (Harman, 1986) and epistemic entrench- 
ment (Gardenfors, 1988). According to the principle of 
minimal change, one should in case of inconsistencies 
make just enough changes in the belief set as are nec- 
essary to reach consistency. The principle of epistemic 
entrenchment arises from the idea that some beliefs are 
less entrenched than others and are therefore more eas- 
ily given up in cases of conflict. Minimal change and 
epistemic entrenchment are well established in many 
formal theories of belief revision (e.g., Alchourron, 
Gardenfors, & Makinson, 1985; Gardenfors, 1988; see 
chapter 5.2 by Rott and chapter 5.3 by Kern-Isberner, 
Skovgaard-Olsen, & Spohn, both in this handbook). 
However, from a psychological point of view, it is not 
clear how minimal change and epistemic entrench- 
ment are implemented cognitively (cf. Elio & Pelletier, 
1997). Therefore, many psychological studies have been 
carried out to understand the psychological processes 
behind these concepts and belief revision. 


3.1 Experimental Findings 

Psychological experiments on belief revision usually fol- 
low the same structure: participants are confronted with 
problems consisting of two (or more) premises and an 
additional fact. This additional fact, however, is incon- 
sistent with the premises. Participants are then told 
that the additional fact is true but that the premises are 
uncertain. The task of the participants is to decide which 
of the two premises they believe less or abandon. One 
of the premises usually contains a general statement, 
such as a conditional (e.g., “If p then q”) or a universally 
quantified statement (e.g., “All As are Bs” or “No As are 
Bs”). Let us illustrate this with an example from condi- 
tional reasoning: 
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If you meditate in the evening, then you fall asleep more 
easily. 
You meditate in the evening. 


But: You do not fall asleep more easily. 


You will have noticed that the premises are the same 
as in MP. The additional fact, however, is not the con- 
clusion that follows from the MP inference. It is even 
inconsistent with the set of premises. Participants thus 
have to indicate which premise they abandon: the con- 
ditional or the categorical one. In a similar way, belief 
revision can also be tested with MT: 


If you meditate in the evening, then you fall asleep more 
easily. 

You did not fall asleep more easily. 

But: You meditated in the evening. 


Again, participants have to decide whether they want to 
discard the conditional or the categorical premise. 

The experimental findings on belief revision are hetero- 
geneous. The first psychological studies on belief revision 
were conducted with quantified statements. Participants 
had to choose between revising premises with a universal 
quantifier (e.g., “All vertebrates have a backbone”) or with 
particular facts (e.g., “This amoeba is not a vertebrate”). 
They consistently chose to revise the particular fact rather 
than the premise with the universal quantifier (e.g., Rev- 
lin, Cate, & Rouss, 2001; Revlis & Hayes, 1972; Revlis, Lip- 
kin, & Hayes, 1971). The authors explained their findings 
by arguing that statements with universal quantifiers can 
be seen as general laws and are therefore more entrenched 
and believed more. However, years later, other studies 
had opposite results. In an influential study, Elio and Pel- 
letier (1997) created conditional inference problems and 
found that the conditional premise—which is also a gen- 
eral law—is abandoned more readily than the categorical 
premise. This effect was more pronounced for MP than 
for MT. The authors argue that the conditional is revised 
more often because if-then “regularities” describe inter- 
dependences between particular facts and are therefore 
more uncertain (Elio & Pelletier, 1997). Elio and Pelletier’s 
findings have been replicated several times (e.g., Byrne 
& Walsh, 2005, experiment 1; Elio, 1997, experiment 1; 
Politzer & Carles, 2001). 

One explanation for these conflicting findings is the 
premises’ content. Studies finding a preference for believ- 
ing the conditional or universal premise used content- 
rich material, for which the reasoner has background 
knowledge. By contrast, studies finding a preference for 
disbelieving the conditional or universal premise used 
unfamiliar content, for which the reasoner does not have 
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prior knowledge. This was tested by Byrne and Walsh 
(2005). In two experiments, they varied the reasoner’s 
familiarity with the conditional premise by creating con- 
ditionals for which the reasoner had no knowledge (“If 
the ruin was inhabited by Pings it had a forcefield sur- 
rounding it”) or had knowledge (“If water was poured 
on the campfire the fire went out”). After presenting 
to the participants the unquestionable fact, they had 
to write down which premise they revised. When the 
content was unfamiliar, participants preferred to revise 
the conditional premise. But when the content was 
familiar, participants preferred to revise the categorical 
premise. Similarly, in another study, Politzer and Carles 
(2001) varied the plausibility of conditionals and found 
that more plausible conditionals were revised less often 
than less plausible ones. Correspondingly, Wolf, Rieger, 
and Knauff (2012) also found that conditional premises 
uttered by less trustworthy sources are revised more often 
than those uttered by more trustworthy ones. Along 
these lines, the number of disablers affects belief revi- 
sion too. Elio (1997) constructed belief revision problems 
with conditionals that differed in their number of dis- 
ablers (including the ones from Cummins et al., 1991). 
She found that the conditional premise was believed less, 
and abandoned more often, for conditionals with many 
disablers compared to conditionals with few disablers. 

Overall, these studies show that human belief revi- 
sion depends on the content of the premises and the 
trustworthiness of the information source. Revlis’s 
assumption that premises containing general rules, such 
as conditionals, are more entrenched because they are 
lawlike applies only as long as this general rule is known 
by, and plausible for, the reasoner. When the conditional 
or universal premise is unfamiliar instead, then reason- 
ers have no problems in revising such beliefs. In other 
words, entrenchment depends on the knowledge base 
of the reasoner (Byrne & Walsh, 2005; Politzer & Carles, 
2001). However, this content-dependent approach can- 
not explain all revision choices. For instance, Elio and 
Pelletier (1997) found that the preference for revising 
the conditional premise instead of the categorical one 
was more pronounced for MP than for MT. This effect 
has also been replicated several times (e.g., Khemlani & 
Johnson-Laird, 2011; Revlin et al., 2001). There are thus 
additional, inference-dependent processes involved in 
belief revision. We present some explanations for this 
MP-MT asymmetry in the next section. 


3.2 Spatial Belief Revision 
The empirical findings on belief revision we have pre- 
sented so far were concerned with conditionals. This is 
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understandable because “if-then” statements are closely 
related to our beliefs about the world. Moreover, the 
experimental paradigm follows the philosophical tradi- 
tion in which an inconsistency arises between an addi- 
tional fact and the conclusion that can be derived from 
a set of premises. But in everyday life, people are often 
forced to change other kinds of beliefs. For instance, 
when following directions or trying to orient themselves 
in foreign cities, people also have to change beliefs about 
spatial location when they are lost. 

Our group was the first that explored belief revision 
processes in the domain of spatial reasoning. In spatial 
reasoning, people infer new spatial relations between 
objects in space from information that is already given 
(Knauff, 2013; Ragni & Knauff, 2013). But what happens 
if new information disagrees with the prior information 
about the objects’ locations in space? In our belief revi- 
sion paradigm, participants get two or more premises 
describing the spatial relations between a set of objects. 
Afterward, an additional fact is presented that conflicts 
with the spatial arrangement consistent with the prem- 
ises. Participants have to decide which objects they relo- 
cate in order to reach consistency. For example: 


A is to the left of B. 
B is to the left of C. 
But: A is to the right of C. 


The two premises allow the construction of the linear 
spatial arrangement A—B-C. However, the contradictory 
fact says that A is to the right of C. There are two ways 
to solve this inconsistency: either moving A to the right 
of C or moving C to the left of A. Following the psycho- 
logical and linguistic tradition on the functional asym- 
metry between arguments, A is called the to-be-located 
object (LO) and C the reference object (RO) (Landau & Jack- 
endoff, 1993). The empirical findings show that partici- 
pants prefer to relocate the LO instead of the RO (Knauff, 
Bucher, Krumnack, & Nejasmic, 2013), although this 
effect can be modulated by properties of the objects 
such as size and movability (Nejasmic, Bucher, & Knauff, 
2015). 


4. Psychological Theories of Defeasible Reasoning 
and Belief Revision 


So far, we have described how defeasible reasoning and 
belief revision are investigated in psychology and given 
an overview of the main experimental findings. Now 
we want to give an overview of the current psychologi- 
cal theories of defeasible reasoning and belief revision. 
This will, hopefully, facilitate the understanding of the 
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relatively complex (and not always consistent) pattern 
of effects and provide insights into the cognitive pro- 
cesses behind defeasible reasoning and belief revision. 


4.1 The Theory of Mental Models 

The theory of mental models (Johnson-Laird, 2010; 
Johnson-Laird & Byrne, 1991; see chapter 2.3 by Johnson- 
Laird, in this handbook) describes human reasoning as a 
semantic process where people use the meaning of prem- 
ises to construct internal mental representations—the 
models—of what would be the case if the premises were 
true. These mental models are inspected and evaluated 
in order to find information that is not already given 
in the premises. The construction of mental models fol- 
lows the principle of truth, meaning that only what is 
true is represented. Therefore, following the postulates 
of classical logic, a conditional “If p then q” allows the 
construction of the following mental models: 


P q 
not-p q 
not-p  not-q 


A conclusion is considered true if it follows necessar- 
ily from the premises. A conclusion is considered false 
if there is a counterexample to it, that is, a model in 
which the premises are true but the conclusion is false. 
For instance, in the models above, we can see that if p is 
true, q is also true. There is no model in which p is true 
but q is false. Therefore, the MP conclusion is true. Along 
the same lines, also the validity of MT and the invalidity 
of AC and DA can be shown. 

Furthermore, the model theory considers the restric- 
tions from working memory. It is postulated that people 
do not consider all mental models from the beginning. 
Instead, they only construct one mental model, the 
explicit mental model p&q, which represents the infor- 
mation explicitly provided in the conditional: 


Pq 


All other possible models (not-p and q, and not-p and 
not-qg) are only represented implicitly, denoted by the 
three dots. Only if required by the task are these implicit 
models fleshed out into fully explicit models. Several 
experiments have shown that people prefer to construct 
specific models (preferred mental models) while ignoring 
other possibilities that are also consistent with the prem- 
ises (neglected mental models) (Jahn, Knauff, & Johnson- 
Laird, 2007; Knauff, 2013). Moreover, the sequence of 
models generated is not random but follows a specific, 
cognitively efficient model variation mechanism (Ragni 
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& Knauff, 2013). This theory has recently been fun- 
damentally revised and extended to account for ordi- 
nary people’s reasoning beyond deduction (Hinterecker, 
Knauff, & Johnson-Laird, 2016; Johnson-Laird, Khem- 
lani, & Goodwin, 2015b). 

Defeasible reasoning Mental model theory accounts 
for defeasible reasoning by assuming that reasoners 
introduce disablers as additional antecedents (Byrne 
et al., 1999; Johnson-Laird & Byrne, 2002). Imagine, for 
instance, that a reasoner is confronted with the condi- 
tional “If Jack goes to bed late, then he will be tired” and 
considers the disabler that he can sleep in. This results in 
the following mental model: 


bed late sleepingin tired 


Because reasoners know that after sleeping in, people 
are usually not tired, they will additionally construct the 
following models, based on general knowledge: 


sleeping in not tired 
not sleeping in tired 
not sleeping in _ not tired 


Thus, when people have to decide whether g follows from 
p, they combine the models from background knowledge 
with the models of the premises. However, the combina- 
tion of these models would yield a contradiction: 


bed late sleepingin tired not tired 


Therefore, Johnson-Laird and Byrne (2002) propose the 
principle of pragmatic modulation: in such cases, the model 
from prior knowledge has priority, allowing reasoners to 
conclude that, if Jack goes to bed late but is able to sleep 
in, he will not be tired. The model theory thus describes 
the consideration of disablers in terms of counterexample 
search in general knowledge. The impact of the frequency 
of exceptions is accounted for by assuming that the prob- 
ability of an event can be inferred from the proportion 
of models in which the event occurs (e.g., Johnson-Laird, 
2001; Johnson-Laird, Legrenzi, Girotto, Legrenzi, & Cav- 
erni, 1999). For instance, when reasoning about the condi- 
tional “If Jack goes to bed late, then he is tired,” reasoners 
may construct the following mental models, containing 
the chances that q actually follows from p: 


bed late tired 3 
bed late nottired 1 
not bed late tired 2 
not bed late nottired 4 


In this case, there are three chances that Jack is tired 
out of four chances that he goes to bed late (see also 
Johnson-Laird, Khemlani, & Goodwin, 2015a). 
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Belief revision According to mental model theory, 
belief revision happens in three steps (Johnson-Laird, 
Girotto, & Legrenzi, 2004; Legrenzi & Johnson-Laird, 
2005): first, participants try to construct a mental model 
where all premises are satisfied. If this is not possible, 
participants infer that the set of premises is inconsistent. 
Second, participants try to solve the inconsistency. If the 
contradicting fact only conflicts with a single premise, 
then this premise is disbelieved. However, if the contra- 
dicting fact conflicts with more premises, then people 
will revise the premise whose model is in conflict with 
the contradictory fact or that fails to represent the fact. 
This process is called the mismatch principle (Johnson- 
Laird et al., 2004). For instance, in the case of MP, people 
are confronted with the conditional “If p then q,” the cat- 
egorical premise p, and the contradictory fact not-q. This 
contradictory fact is clearly in conflict with the explicit 
model from the conditional premise (p q). Therefore, the 
mismatch principle predicts that, for MP, the conditional 
premise will be abandoned. In MT, by contrast, people 
are confronted with the conditional “If p then gq,” the 
categorical premise not-q, and the contradictory fact p. In 
this case, p is not in conflict with the explicit model and 
thus the categorical premise will be abandoned instead of 
the conditional. This explains why the conditional prem- 
ise is more often disbelieved in MP than MT. Finally, after 
belief revision, participants search for a diagnostic expla- 
nation that resolves the inconsistency they observed. For 
instance, in our example about the failed meditation, 
such an explanation could be “It was the wrong medita- 
tion and therefore he did not fall asleep” (see Legrenzi & 
Johnson-Laird, 2005). 


4.2 Probabilistic Theories 

Probabilistic theories assume Bayesian probability the- 
ory as the right norm for human reasoning (Evans, 2012; 
Oaksford & Chater, 2007; see chapter 4.5 by Chater & 
Oaksford and chapter 4.6 by Oberauer & Pessach, both 
in this handbook). Proponents of probabilistic theo- 
ries of reasoning assume that people treat conditionals 
probabilistically by understanding the probability of a 
conditional, P(if p then q), as the conditional probabil- 
ity of q given p, P(q|p)—a relationship known as “the 
Equation” (Edgington, 1995; Evans & Over, 2004). The 
conditional probability can be calculated by perform- 
ing the Ramsey test (Ramsey, 1929/1990; see also Evans, 
Handley, & Over, 2003). According to the Ramsey test, 
people first suppose that p holds. On the basis of this 
hypothetical belief, people then estimate the probability 
of q. For instance, in the case of “If Jack goes to bed late, 
then he will be tired,” reasoners first assume that Jack 
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goes to bed late. Then they start thinking about how 
probable it is that a person actually feels tired after going 
to bed late (P(p &q)) and how probable it is that this is 
not the case (P(p &not-q)). The higher P(p &4q) is relative 
to P(p &not-q), the higher is the conditional probability 
P(q|p) (Evans & Over, 2004; Evans et al., 2003). And the 
higher this conditional probability, the more probable it 
is that an MP conclusion is accepted (Oaksford, Chater, & 
Larkin, 2000). 

Defeasible reasoning The notion of conditional prob- 
abilities captures defeasible reasoning directly. In fact, 
nonmonotonic and defeasible reasoning were the pur- 
poses probabilistic theories were developed for in the 
first place (Oaksford & Chater, 2013). All informa- 
tion that influences the conditional probability affects 
people’s acceptance of conclusions. Therefore, disablers 
affect reasoning: disablers lower the conditional proba- 
bility P(q|p), and as a consequence, also fewer MP and MT 
conclusions are drawn (Weidenfeld, Oberauer, & Hornig, 
2005). In a similar way, alternatives affect inferences too: 
alternatives lower the conditional probability P(p|q), 
and therefore fewer AC and DA conclusions are drawn 
(Verschueren, Schaeken, & d’Ydewalle, 2005). Further, 
given that probabilistic theories do not assume mental 
representations of concrete p-, q-, not-p-, or not-q-cases, 
they can explain why in addition to the number of dis- 
ablers, also the overall frequency of exceptions affects 
inferences. Probabilistic approaches do not require 
people to construct concrete representations from gen- 
eral knowledge to be aware that p can happen without q. 
Instead, the effect of disablers is operationalized by the 
probability P(p &not-q), which is directly related to the 
frequency of exceptions. 

Belief revision On the normative side, there are many 
accounts that explain belief change, for instance, through 
Bayesian conditionalization. However, from a psycho- 
logical point of view, these accounts may be problem- 
atic (see Oaksford & Chater, 2013). Therefore, further 
research is needed to understand how people change 
their probability distributions after learning new infor- 
mation. For instance, Wolf, Rieger, and Knauff (2012) 
provided an account of how probabilities could describe 
the asymmetry in belief revision for MP and MT. The 
main assumption is that the contradictory fact changes 
the probabilities of the premises. For instance, in the 
case of belief revision after MP, the fact not-g is presented 
as true; therefore, P(not-q) = 1. This implies that P(q) = 0, 
and consequently, P(q | p) = P(if p then q) = 0. However, in 
the case of belief revision after MT, the fact p is presented 
as true; therefore, P(p) = 1. Consequently, P(q | p) = P(if p 
then q) = q. The comparison of these probabilities shows 
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why the conditional premise is often believed less for 
MP than MT. 

The influence of plausibility and trustworthiness on 
belief revision is described by probabilistic theories through 
the role of long-term memory. The main idea is that long- 
term memory about the content of a conditionals affects 
the corresponding conditional probability (Oaksford & 
Chater, 2013). Similarly, also knowledge about plausibil- 
ity and trustworthiness is part of our knowledge system. 
Therefore, participants give lower probabilities to implau- 
sible conditionals and to conditionals uttered by untrust- 
worthy sources. As a consequence, these are believed less 
and abandoned more readily (cf. Wolf et al., 2012). 


4.3 Dual-Process Theories 

Dual-process theories propose that people use two kinds 
of thinking or modes of processing that are employed 
depending on factors such as tasks, time constraints, 
or cognitive capacity (Evans, 2008; Evans & Stanovich, 
2013; see chapter 2.4 by Fiedler, Prager, & McCaughey, 
this handbook). People can reason heuristically, fast, 
and effortlessly (known as “System 1” or “Type 1” pro- 
cesses), and they can reason analytically, slowly, and 
dependently on working memory (known as “System 2” 
or “Type 2” processes) (e.g., Evans, 2008; Kahneman & 
Frederick, 2005). It is argued that System 1 serves as 
a default system, whose responses can be either over- 
ridden or supported by System 2 (Evans & Stanovich, 
2013). 

Defeasible reasoning As we have shown, disablers 
and alternatives can affect reasoning in two ways: either 
as single instances or through the frequency of p & not- 
q- and not-p &q-cases. Verschueren et al. (2005) argue 
that both aspects are important and can be unified in 
a dual-process approach. They argue that the consider- 
ation of single disablers and alternatives is an analytic 
process—as proposed by mental model theory. In con- 
trast, the consideration of the frequency information 
is considered a heuristic process, in which the overall 
likelihood of q given p and of p given q matters—as pro- 
posed by the probabilistic theories. Verschueren et al.’s 
(2005) dual-process model has been tested experimen- 
tally and corroborated several times (e.g., Markovits, 
Brisson, & de Chantal, 2015, 2017). As predicted by 
dual-process theories, fast responses are best explained 
by the likelihood of q given p (for MP and MT infer- 
ences) or the likelihood of p given q (for AC and DA 
inferences), slow responses are best described by the 
availability of disablers or alternatives, and conclusions 
based on likelihoods can be overridden by specific dis- 
ablers and alternatives. 


Defeasible Reasoning and Belief Revision in Psychology 


Belief revision A dual-process approach to belief revi- 
sion can be found in Wolf and Knauff (2008). In their 
experiments, they varied the probability of the condi- 
tional premise (0%, 50%, or 100%). They found that when 
the conditional premise was highly probable or highly 
improbable, participants used this probability informa- 
tion as a heuristic for belief revision: they believed the 
conditional premise more for highly probable condition- 
als but believed the categorical premise more for highly 
improbable conditionals. However, when the probabil- 
ity of the conditional was around 50%, people reasoned 
more analytically, and the mismatch principle from 
mental model theory predicted belief revision choices: 
the conditional premise was believed more for MT than 
for MP. 


5. Defeasible Reasoning or Belief Revision? 


So far, we have treated defeasible reasoning and belief 
revision separately. The main reason is that both func- 
tions are often discussed within different theoretical 
(normative) frameworks and communities. In defea- 
sible reasoning, conclusions are withdrawn in light of 
new evidence, while in belief revision, the new evidence 
makes people abandon previous beliefs (in the psychol- 
ogy lab typically operationalized as premises). However, 
a germane question is whether defeasible reasoning 
and belief revision do indeed rely on distinct cognitive 
processes—a crucial descriptive question from a psycho- 
logical point of view. In fact, the psychological litera- 
ture on defeasible reasoning and belief revision shows 
that both depend on the content of the premises. In 
defeasible reasoning, many disablers and unbelievable 
content increase people’s willingness to reject conclu- 
sions from conditional premises. In a similar way, in 
belief revision, conditionals with many disablers and 
implausible content are revised more often than those 
with fewer disablers or plausible content. These similari- 
ties were noticed by Dieussaert et al. (2005). They con- 
ducted a belief revision experiment using conditionals 
with many and with few disablers and asked the partici- 
pants twice about their belief in the conditional prem- 
ise: before presenting the contradicting fact and after. 
They found no significant differences in the believ- 
ability ratings before and after the introduction of the 
inconsistency. This shows that people’s disbelief in the 
conditional premise was not the outcome of a process of 
belief revision but a product of people’s consideration of 
disablers. Dieussaert and colleagues therefore argue that 
people’s withdrawal of conclusions in conditional rea- 
soning and in belief revision are “two sides of the same 
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coin” (Dieussaert et al., 2005, p. 29; see also Gardenfors, 
1990). 

At first sight, the findings by Dieussaert et al. (2005) 
seem counterintuitive. In belief revision, participants 
abandon premises in the search for consistency, while 
in defeasible reasoning, there is no inconsistency at all. 
Nevertheless, there is evidence suggesting that this dis- 
tinction is only a result of differing experimental para- 
digms. For instance, Walsh and Johnson-Laird (2009) 
showed that when participants are asked to explain 
inconsistencies, they rarely just negate the conditional 
premise but rather explain inconsistencies by naming 
disablers that question the causal link between ante- 
cedent and consequent (see also Johnson-Laird et al., 
2004). In a later experiment, Khemlani and Johnson- 
Laird (2011) showed that in case of inconsistencies, 
participants prefer to give explanations instead of direct 
refutations of the premises. That is, participants seem 
to resolve inconsistencies by finding disablers, without 
the need of a revision in the strict sense. Similarly, also 
Politzer and Carles (2001) found that when participants 
can choose between completely abandoning one of the 
premises and only casting doubt on them, they prefer 
to cast doubt. These findings suggest that under every- 
day circumstances, people do not spontaneously aban- 
don existing beliefs but instead try to find explanations 
for inconsistencies in the set of beliefs. This has conse- 
quences for the relation between defeasible reasoning 
and belief revision. An example: 


If you meditate in the evening, then you fall asleep more 
easily. 
You meditate in the evening. 


But: You do not fall asleep more easily. 


In a belief revision paradigm, participants have to 
choose which premise they reject. However, in everyday 
life, participants would rather try to explain the incon- 
sistency by the introduction of a disabler: 


If you meditate in the evening, then you fall asleep more 
easily. 

If it is the wrong meditation, then you do not fall asleep more 
easily. 

You meditate in the evening. 


You do not fall asleep more easily. 


Now the inference resembles one of defeasible reason- 
ing: by introducing the disabler, the formally inconsis- 
tent fact turns into the “conclusion” that participants 
would have accepted in a defeasible reasoning paradigm. 
The search for explanations plays a vital role in these 
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cognitive processes, albeit one that is not so frequently 
considered in normative accounts of defeasible reason- 
ing and belief revision. 


6. Concluding Remarks and Further Perspectives 


Many psychological studies help us to understand human 
defeasible reasoning and belief revision. Still, there is 
need for further research. For instance, the principle 
of truth of mental model theory has difficulties in cap- 
turing people’s consideration of p & not-q-cases during 
reasoning (Geiger & Oberauer, 2010). Similarly, probabi- 
listic theories are only computational theories (accord- 
ing to Marr’s, 1982, terminology) that do not describe 
the cognitive processes behind reasoning (Oaksford & 
Chater, 2003). However, cognitive theories of reasoning 
must explain not only what is computed but also how it 
is computed in the human mind (Hinterecker, Knauff, 
& Johnson-Laird, 2019; see the introductory chapter by 
Knauff & Spohn, this handbook). Dual-process theories 
have been criticized for not describing qualitative dif- 
ferences but only quantitative differences in reasoning 
(Osman, 2013) and for just exemplifying “the backwards 
development from precise theories to surrogates” (Gige- 
renzer, 2011, p. 739; but see chapter 1.2 by Evans, this 
handbook). 

What remains clear from our review is that still more 
research is needed to understand the cognitive processes 
that underlie human defeasible reasoning and belief 
revision. Here, more collaboration between psychology, 
philosophy, and AI could be helpful. There are many 
normative and formal theories on defeasible reasoning 
(e.g., Kraus, Lehmann, & Magidor, 1990; Spohn, 2012; 
see chapter 5.3 by Kern-Isberner, Skovgaard-Olsen, & 
Spohn, this handbook) and belief revision (e.g., Alchour- 
ron etal., 1985; see chapter 5.2 by Rott, in this handbook), 
which can be helpful in cognitive psychology, too. At the 
same time, however, these formal theories should also 
consider psychological findings and adapt their formal- 
isms to the actual cognition and behavior of real people. 
We are well aware of the different views on this is—ought 
problem (see Knauff and Spohn’s introduction to this 
handbook). Yet, it is obvious that there are still too few 
links between the normative and the descriptive work on 
defeasibility and belief revision. On the one hand, many 
of the formalisms from philosophy and AI have not been 
tested regarding their capability to model actual human 
reasoning (cf. Ragni, Eichhorn, Bock, Kern-Isberner, & 
Ping Ping Tse, 2017). On the other hand, psychologi- 
cal research has neglected nonmonotonicity for a long 
time. Therefore, psychologists have often prematurely 
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considered people irrational, just because they did not 
follow the norms of classical logic. In fact, many of the 
psychological studies we have reported here do not even 
use the technical term “defeasible reasoning,” although 
they have investigated the circumstances under which it 
is rational to withdraw otherwise valid conclusions. To 
avoid such dangers of disciplinary division, we need a 
more interdisciplinary approach. Ideally, this research 
combines the strengths of psychology, philosophy, and 
AI to obtain a broader understanding of how a cogni- 
tive system, be it real or abstract, human or machine, 
can flexibly respond to new information and therefore 
change its belief system in a rational way. 


Note 


1. Geiger and Oberauer (2007) only investigated the relation 
between the number of disablers and the frequency of excep- 
tions. However, it is also conceivable that the frequency of 
alternatives affects inferences. 
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Summary 


Unlike persuasion, the study of “rational argument” 
asks what convinces and what should convince (see, e.g., 
Hahn & Oaksford, 2012). It combines normative and 
descriptive concerns, both within theory development 
and the study of behavior. In this chapter, we show the 
centrality of norms, discuss how norms have shaped 
empirical research, and highlight how descriptive work 
has informed normative developments, showing the 
tight coupling between argumentation, reasoning, and 
rationality. Finally, we identify future directions for 
research. 


1. What Are Arguments; Why Study Them? 


We distinguish two relevant senses of “argument”: argu- 
ments as objects and arguments as a kind of social pro- 
cess. Arguments-as-objects are sets of statements: claims 
or conclusions supported by premises.' The strength of 
support varies. Arguments can be deductive: the conclu- 
sion follows necessarily; if one believes the premises, 
one denies the conclusion on pain of inconsistency. 
Arguments can also be inductive or abductive: in either 
case, the inference is ampliative in the sense that the 
conclusion may be false even if the premises are true. 
Often such inference is cast as defeasible (presumptive): 
an argument that “may be strong enough to provide evi- 
dence to warrant rational acceptance of its conclusion, 
given that its premises are acceptable” (Walton, Reed, & 
Macagno, 2008, p. 1). But acceptance of the conclusion 
is tentative: it may be overturned by further informa- 
tion (Walton et al., 2008). It is subject to debate to what 
extent these different types require different normative 
frameworks. In the remainder, we use the term “induc- 
tive” broadly to refer to ampliative arguments, setting 
aside (unless otherwise indicated) differences between 
statistical generalizations (“inductive” in a narrower 
sense), arguments from evidence to causes (e.g., “abduc- 
tion”), and defeasible arguments. 


Arguments-as-processes are dialogues in a social con- 
text: dialectical exchanges, with discussants propos- 
ing and opposing claims, say, to convince someone of 
the truth of some claim or persuade someone to per- 
form some action (for a typology, see Walton, 2008). In 
recent decades, much argumentation research has cen- 
tered, explicitly or implicitly, on the extent to which 
arguments-as-objects can be understood without the 
wider vantage point of arguments-as-processes. 

Psychology has tended to focus on arguments-as- 
objects. In particular, the psychology of reasoning has 
investigated how people derive, or endorse, conclusions, 
typically of deductive and inductive arguments (for a sur- 
vey, see Manktelow, 2012). In mainstream psychology of 
reasoning, studies have given relatively little attention to 
social and conversational contexts (although see Evans, 
Neilens, Handley, & Over, 2008; Girotto, Kemmelmeier, 
Sperber, & van der Henst, 2001; Mercier & Sperber, 2011, 
2017; Sperber, Cara, & Girotto, 1995; Stevenson & Over, 
2001). Yet, if one closely considers how people change 
their beliefs upon hearing certain premises, one must 
eventually consider the contexts in which such premises 
might be uttered. For example, if one tries to understand 
why participants deviate from the logical prescriptions 
of modus tollens, one soon sees the need to consider 
why a particular premise might be uttered in real life 
(see, e.g., Hahn & Oaksford, 2012). 

An undue focus on arguments-as-processes suffers 
complementary difficulties. Historically, it was argued 
that so-called fallacies of argumentation—such as “cir- 
cular arguments” or “question-begging”—could only be 
adequately understood as fallacious by considering them 
as argumentative “moves” in a dialectical context (see, 
e.g., Hahn, 2011, and references therein). Hence, a circu- 
lar argument such as “God exists, because God exists” is 
viewed as fallacious because it violates a procedural com- 
mitment such as a requirement to meet one’s “burden 
of proof.” This renders the argument weak even though 
it is actually deductively valid and, in that sense, strong. 
However, such explanation by procedural appeal to the 
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burden of proof is, arguably, itself question-begging, 
because the argument can only fail to meet that bur- 
den of proof because it is weak. This, however, means 
that some independent measure of the strength of the 
argument’s content is required. In other words, the pro- 
cedural failure presupposes the weakness it is trying to 
explain (Hahn, 2011; Hahn & Oaksford, 2007). 

Any satisfactory account of argument thus ulti- 
mately requires consideration of the properties of both 
arguments-as-objects and arguments-as-processes. In 
other words, to understand when an argument should 
convince, we need two distinct, but interconnected, sets 
of norms. 


2. Norms 


“Good argumentation” is widely recognized as a desir- 
able goal, for instance, by education systems (Crowell & 
Kuhn, 2014; Kuhn & Crowell, 2011). In defining “good- 
ness,” one can usefully distinguish two types of norm 
(Hahn & Oaksford, 2012): first, procedural norms, aimed 
at regulating discussants’ behavior to achieve consensus 
and, second, epistemic norms, aimed at regulating struc- 
ture and content to achieve truth or justified belief. 

Procedural accounts have drawn inspiration from 
natural-language pragmatics (Walton, 2008). Understood 
abstractly, arguments comprise the following procedures: 
proponents secure commitments from their opponents to 
advance their own case and exploit weaknesses in their 
opponents’ argument to undermine their opponents’ 
case (Walton, 1989). An influential example is pragma- 
dialectics, which posits different stages of argumentative 
dialogues and a set of rules to regulate behavior at each 
stage (van Eemeren & Grootendorst, 1984, 1992, 1995, 
2004). An intuitive rule is the “burden of proof” rule, 
according to which proponents are expected to offer evi- 
dence to support their standpoints. 

Argument content plays a deeper role in epistemic 
norms. Some epistemic norms are familiar from the 
psychology of reasoning. A central normative system is 
classical logic, which enforces consistency through its 
definition of logical validity: that the truth of the prem- 
ises guarantees the truth of the conclusion. A “good 
argument,” from the perspective of classical logic, is 
a valid argument. This does not, however, suffice for 
ampliative argument, leaving the majority of everyday 
informal argument unaccounted for. 

Probability theory generalizes classical logic and adds 
to logical consistency the notion of probabilistic coher- 
ence. This has been exploited in the body of work known 
as “Bayesian Argumentation” (Hahn & Oaksford, 2007; 
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Zenker, 2012). A good argument, from this perspective, 
is one in which the premises mandate an increase in 
the probability (degree of belief) we assign to the con- 
clusion. This framework formally explains so-called 
fallacies of argumentation (such as circular arguments, 
mentioned above; see Hahn & Oaksford, 2006, 2007; 
on the catalogue of fallacies, see, e.g., Woods, Irvine, & 
Walton, 2004), thus filling a long-standing theoretical 
“gap” (Hamblin, 1970). A probabilistic approach can be 
extended to include utilities, invoking decision theory 
to capture considerations of argument strength (see, e.g., 
Corner, Hahn, & Oaksford, 2011; Evans et al., 2008). 

In the literature, there have been tensions between 
procedural and epistemic approaches, illustrated by Wal- 
ton et al.’s (2008) critical discussion of probabilities as a 
model for argumentation and by Bhatia and Oaksford’s 
(2015) test of competing procedural and probabilistic 
predictions for ad hominem arguments. Nevertheless, 
argumentation is a complex phenomenon, and differ- 
ent normative systems might be valuable for different 
facets of argumentation. Uniquely, procedural rules, for 
instance, advise participants how to behave to secure 
consensus and can be understood as ethical prescrip- 
tions (Christmann, Mischo, & Flender, 2000; Schreier, 
Groeben, & Christmann, 1995). Ultimately, the two sys- 
tems are complementary, not least because procedural 
rules will likely have to appeal to epistemic norms for 
their foundation (Corner & Hahn, 2013). 

Presently, it is not clear that procedural rules have 
deep normative force (Corner & Hahn, 2011; Collins & 
Hahn, 2016; Walton et al., 2008). There are, however, 
deep justifications for the probabilistic approach to 
argumentation: mathematical arguments for the opti- 
mality of Bayesian inference (Leitgeb & Pettigrew, 2010a, 
2010b; Pettigrew, 2016; Rosenkrantz, 1992) and its rela- 
tionship with belief accuracy or truth. Ultimately, at 
least one factor in an appropriate foundation for proce- 
dural rules is likely to be that they maximize opportunities 
for truth to emerge (although it is not necessarily the only 
factor; see recent considerations of virtue epistemology 
in argument, which blur lines between content and pro- 
cess when it comes to quality; Aberdein, 2010; Paglieri, 
2015). Normative systems may help to resolve disputes 
(van Eemeren, Garssen, & Meuffels, 2009) and resolve 
them in accuracy-maximizing ways. Detailed elabora- 
tions of such foundations have yet to be provided. 


3. Empirical Evidence 


Similarly intertwined when it comes to argumenta- 
tion are normative and empirical considerations: the 
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conceptual justification of norms for argumentation 
means that data on argumentation can speak to human 
rationality, about which there is considerable debate in 
cognitive science. At the same time, probing people’s 
adherence to putative normative systems governing 
argumentation means probing the conventional validity 
of these systems, that is, examining how far they under- 
pin the argumentative behavior of typical discussants, 
untrained in analysis or evaluation of argumentation 
(Schellens, Sorm, Timmers, & Hoeken, 2017; van Eeme- 
ren et al., 2009). We next sketch current evidence on 
how people assess arguments and how those data bear 
on the validity of normative systems. 

One source of evidence is naturalistic data: the con- 
ceptual analysis of real-world arguments. Procedural- 
rules theorists have analyzed extended arguments (van 
Eemeren & Houtlosser, 1999). So, too, have Bayesians, 
a notable example being a Bayesian reconstruction of 
the entire body of evidence in one of the United States’ 
most famous court cases (Kadane & Schum, 1996). The 
argumentation-scheme literature has compiled systems 
of generalized “types” of arguments that form the basic 
components of much of everyday informal argument. A 
wide range of different schemes has been proposed (Gars- 
sen, 1997; Hastings, 1962; Kienpointner, 1992; Perelman 
& Olbrechts-Tyteca, 1969; Schellens, 1985; Walton et al., 
2008), with over 50 different schemes distinguished by 
some authors. Some aspects of those typologies may 
arguably be redundant, but these schemes are not yet 
exhaustive, for example, in the domain of causal argu- 
ment (Hahn, Bluhm, & Zenker, 2017). These (general- 
ized) descriptive data have themselves been subjected to 
normative analysis from various viewpoints (e.g., Wal- 
ton et al., 2008), including a probabilistic perspective 
(Hahn & Hornikx, 2016). Such qualitative data suggest 
the usefulness of procedural and epistemic normative 
systems. 

More controlled datacome from cognitive-psychological 
experiments. These experiments tend to manipulate prop- 
erties of arguments and have participants assess the rea- 
sonableness, persuasiveness, or convincingness of the 
arguments on Likert-style scales. In the procedural-rules 
tradition, van Eemeren et al. (2009) have found evidence 
for the conventional validity of procedural rules. In one 
study, participants read arguments containing legitimate 
attacks on a person, disguised fallacious ad hominem 
attacks, and openly abusive ad hominem attacks. Partici- 
pants distinguished among these conditions in judging 
reasonableness (similarly, in the “fairness rules” formula- 
tion, see Christmann et al., 2000; Schreier et al., 1995). 
In the argumentation-schemes tradition, Schellens et al. 
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(2017) found that participants were sensitive to different 
argument parameters for a range of schemes, including 
arguments from authority, analogy, and example. Par- 
ticipants also invoked such parameters in open-ended 
questioning. 

A larger literature has tested whether the Bayesian 
approach to argumentation has conventional validity, 
finding considerable support. The approach is to iden- 
tify probabilistic parameters, manipulate them experi- 
mentally, and assess whether participants are sensitive 
to them. Arguments are typically set in dialogues. Take 
arguments from ignorance. As Hahn and Oaksford 
(2007) show, these arguments can be modeled analo- 
gously to drug testing. For example: 


(1) Drug A is toxic because a toxic effect has been 
observed. 


(2) Drug A is not toxic because no toxic effect has been 
observed. 


Argument (2) is an argument from 
Reconstructed probabilistically, the strength of (1) and 
(2) can be modeled with Bayes’ rule. The crucial para- 
meters are sensitivity (P(Effect | Toxicity)) and specificity 
(P(NoEffect | NoToxicity)). In several tasks, participants 
were influenced by these parameters. 

Participants show the influence of probabilistic par- 
ameters in other common arguments. For instance, a 
Bayesian analysis of slippery-slope arguments—“If you 
allow gay marriage, people will soon want to marry their 
pets”—predicts influence of both probabilities and utili- 
ties, that is, P(OQutcome| Action) and U(Outcome) (see 
Hahn & Oaksford, 2007). Experimental evidence sug- 
gests such influence and links it to cognitive mechanisms 
for categorization (Corner et al., 2011). Similarly, Bayes- 
ian models have been applied to arguments involving 
sources. Inspired by models of testimony, these models 
account well for participants’ assessments of ad hominem 
arguments and arguments from expertise (Harris, Hahn, 
Madsen, & Hsu, 2016; Harris, Hsu, & Madsen, 2012). 

Thus far, there is evidence that participants, at a group 
level, endorse procedural rules, argumentation schemes, 
and probabilistic norms. That they endorse probabilistic 
norms can also be taken as evidence that people approx- 
imate behaviors that have deep conceptual justification. 


ignorance. 


4. Future Directions 


The psychology of argumentation has yet to fully capital- 
ize on a wide range of possible cognitive-psychological 
methods for exploring argumentation. Two promis- 
ing areas are exploration of individual differences and 
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intervention studies to improve argumentation. As we 
will see below, there is already suggestive, but limited, 
evidence. The limitation arises from the fact that we do 
not currently have reliable, theory-neutral measures of 
argumentation skills. Such measures would be challeng- 
ing to develop but would yield rich rewards. 

We turn, first, to the study of individual differences. 
Such studies allow us to probe conventional validity in 
more detail, asking whether norms are endorsed to the 
same extent by all participants and what participant 
characteristics drive differences in endorsement. Perhaps 
more interestingly, studies could also offer evidence for or 
against certain norms, when paired with a reliable mea- 
sure of argumentation skills. In principle, adherence to 
certain norms, or related cognitive skills, should correlate 
with higher scores on measures of argument quality. 

Existing research offers suggestive evidence. Argumen- 
tation skills have largely been assessed using procedural 
norms: for instance, the number of counterarguments 
and rebuttals might be counted (Nussbaum & Asterhan, 
2016) without deep assessment of the content (although 
see Kuhn, 1991, for some more epistemic criteria, albeit 
with unclear validity). Compared with such norms, few 
people attain proficiency (for discussion, see Asterhan & 
Schwarz, 2016). Individual characteristics such as level 
of education (Kuhn, 1991) or epistemological belief 
(Kuhn, Cheney, & Weinstock, 2000; Kuhn, Wang, & 
Li, 2010) predict success to some degree. For instance, 
there are “multiplists,” who view facts as freely chosen 
opinions and tend not to engage in argumentation at 
all, but there are also “evaluativists,” who view facts as 
judgments that can be evaluated against standards of 
evidence (Kuhn et al., 2000; Kuhn et al., 2010). This lit- 
erature could be enriched by considering other norms 
and a theory-neutral definition of skill. 

Another potentially rich source of evidence is 
intervention studies: studies that set out to improve 
argumentative skills. These studies can offer data on 
conventional validity through their control groups. As 
with individual-difference studies, the data are more 
suggestive than decisive. A growing literature demon- 
strates that intervention can increase adherence to pro- 
cedural norms, in particular, increase the frequency of 
attempts to make claims, justify them, respond to other 
arguments, and compare competing arguments (Crow- 
ell & Kuhn, 2011, 2014). Successful interventions range 
from an intensive and complex three-year program with 
children (Crowell & Kuhn, 2014; Kuhn & Crowell, 2011) 
to an hourlong intervention with undergraduate stu- 
dents (Zavala & Kuhn, 2017). Perhaps the most striking 
evidence comes from studies that demonstrate transfer 
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from argumentation tasks to other tasks, such as from 
training in mathematical argumentation to scores on 
standardized tests of mathematics and reading (Nuss- 
baum & Asterhan, 2016). Such assessments are theory 
neutral regarding argumentation. Thus far, there do not 
seem to be equivalent studies testing interventions based 
on argumentation schemes or Bayesian norms. But there 
is no reason to consider such interventions impossible, 
and they would pave the way for contrastive studies. 

Strikingly, the empirical literature does not seem to 
have asked whether particular norms help resolve dis- 
putes. While this question is sometimes considered 
conceptual (van Eemeren et al., 2009), it seems straight- 
forward to imagine its empirical study. There is already a 
large literature on group judgment and decision making, 
exploring both how people interact to reach a single judg- 
ment or decision (for a review, see Kerr & Tindale, 2004) 
and the extent to which argument improves performance 
(e.g., Mercier, Trouche, Yama, Heintz, & Girotto, 2015). 
Such methods could be adapted to test whether compli- 
ance with sets of norms improves the likelihood or effi- 
ciency of arriving at a consensus. The consensus answers 
could be assessed in the usual way, to test whether these 
norms increase success in tracking the truth. 


5. Conclusions 


The psychology of argumentation offers valuable, if 
limited, data. There is evidence that people implicitly 
endorse both procedural and epistemic norms for argu- 
mentation. Argumentation seems rational in the sense 
that it is a considered, rule-governed behavior. Moreover, 
data suggest that (at least some) people approximate 
probabilistic norms when evaluating arguments—behav- 
ior that has deeper conceptual justification and a stron- 
ger claim on rationality. 

Nevertheless, much remains to be done. Probabilis- 
tic norms, in particular, lend themselves to constrained 
models, which offer substantial advantages over existing 
approaches, where much is left unspecified, rendering 
the models hard to falsify (for discussion, see Schellens 
et al., 2017). Theories and models of argumentation can, 
likewise, link pragmatic reasoning to specific forms of 
argument, offering increased specificity to the study of 
natural-language pragmatics (Macagno & Walton, 2013). 

In sum, we have considered the merits of studying 
the psychology of argumentation, its connections with 
the psychology of reasoning, and the central feature they 
both share: the interdependence of descriptive and nor- 
mative considerations. 
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Note 


1. Sometimes, “arguments-as-objects” may also be used to refer 
to just the premises or “reasons” themselves. It should be clear 
from the context which use is intended. 
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Summary 


Drawing on the work of informal logicians, speech com- 
munication scholars, rhetoricians, critical-thinking the- 
orists, empirical and theoretical linguists, cognitive and 
experimental psychologists, dialogue theorists and dia- 
lectical logicians, and, more recently, computer scientists 
and artificial intelligence (AI) theorists, argumentation 
theory is a large and growing network of enquiry into 
natural-language argument and its tie to reason-giving 
and inference. This entry concentrates on theoretical 
developments from the late 1950s to the present day. 
My remarks are organized under the following headings: 
(1) Argument, (2) Reasoning, (3) Toulmin and the Rhe- 
torical Turn, (4) Hamblin Contra the Logicians, (5) The 
Revival of Fallacy Theory, (6) Informal Logic Contra the 
Textbooks, (7) Pragma-Dialectics, and (8) More Recent 
Developments. The coverage is not complete. For want 
of space, I omit developments in speech communica- 
tion, rhetoric, and critical thinking, concerning which 
see Blair (2019). 


1. Argument 


Logic was founded as a theory of argument. Aristotle’s 
syllogistic logic was the first nonmonotonic, relevant, 
and paraconsistent logic, and some decent approxima- 
tion to an intuitionist one, each of which today is termed 
“nonclassical” (Woods, 2015/2018). Aristotle was also 
the originator of a metalogic in which he produced a 
near-perfect proof of the semi-decidability of validity in 
syllogistically formulated arguments’ (Corcoran, 1972). 
The point of mentioning this here is that in most of the 
approaches to argument under review here, decidability 
proofs are neither sought nor possible. 

Aristotle also placed the analysis of fallacies— 
paralogismoi—on the front burner of logical theory. In 
1879 and following, however, logic changed course. 
Its new purpose was to provide secure foundations for 


number theory in a newly created second-order quanti- 
ficational system in which fallacy theory had no place” 
(Frege, 1884/1950). 

As we have it now, argumentation theory is an investi- 
gation of how in various contexts conclusions are arrived at 
from premisses. Conclusions take different forms—about 
what is true or false, probable or improbable, plausible or 
implausible, persuasive or unpersuasive, about what is best 
to do, most prudent to do, and so on. Although human 
conclusion-drawing need not be argumentally structured, 
it is widely believed that premiss—conclusion reasoning 
can insightfully be modeled as premiss—conclusion argu- 
ment (see, e.g., Mercier & Sperber, 2017). 

Two further distinctions call for attention: the first 
recognizes three different ways in which the consequence- 
relation manifests itself—consequence-having (or entail- 
ment), consequence-spotting (or recognition), 
consequence-drawing (or inference). Having is a binary 
relation between premisses and conclusion. Spotting and 
drawing require agents and hence are ternary relations 
(Woods, 2013/2014). It is broadly accepted that a theory 
that works well for entailment also works well for infer- 
ence. In fact, this is seriously mistaken (Harman, 1970). 

While the Trinitarian character of consequence has 
received little recognition in the literature, our second 
distinction, that between an argument as-such and an 
argument in-use, is at least implicitly grasped. An argu- 
ment as-such is a finite sequence of statements whose 
terminal member, the conclusion, is a consequence of 
the other members, its premisses. Arguments in-use 
are what happens to arguments as-such when goal- 
directed agents engage them in support of their various 
ends. Logics of arguments as-such concentrate on the 
consequence-having relation (Hitchcock, 2017, chap- 
ter 7; Woods, 2011). Logics for consequence-spotting 
and for consequence-drawing are agent centered, goal 
directed, and (frequently) dynamically interactive. 

The word “argument” is an open-textured term, sub- 
ject to the loose constraints of family resemblance, hence 


and 
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a term that precludes definition by way of necessary 
and sufficient conditions. The extension of that term is 
large and sprawling, stretching from conversationally 
expressed differences of opinion to counsels’ closing 
summations at trial, newspaper editorials and political 
cartoons, partisan placards, World Trade Organization 
negotiations, and nightly a table screaming matches 
between Bill and Sue. Use of the word “argumentation” 
adds no clarity to this already fuzzy assortment. This is 
just to say that “argumentation” is no improvement as 
a synonym of “argument.” Even so, in some uses, argu- 
mentation is a theory of argument, and in others, it is a 
theory of theories of argument (Blair, 2012, pp. 198-199). 

Rhetorical studies range more broadly, often with 
insightful probes into the give-and-take of arguments at 
their least civilized. Theorists of argument are respon- 
sive to varying standards: they can aim for normative 
accuracy. They can aim for descriptive accuracy. Some- 
times they aim for both. A descriptively accurate theory 
of arguments of kind K is an objectively true account 
of how K-arguments are made and how they play out 
in real life. A normatively accurate theory of arguments 
of kind K is an account of what it takes for a well-made 
K-argument to satisfy what are usually implicit norms 
of goodness. It would be seriously wrong to assume that 
these hoped-for goals are only the ones we see listed 
in the introductory textbooks—deductive validity and 
inductive strength. A glaring omission is the logic of 
abductive reasoning (Gabbay & Woods, 2005). 

Argument kinds make for a loose and somewhat fuzzy 
taxonomy, neither exclusive nor exhaustive, and with 
lots of cross-border traffic. In none of its subdisciplines is 
there anything resembling full coverage of the extension 
of the word “argument.” In most cases, there are features 
of it that even the better accounts ignore or downplay. 
In the dominant approaches, the theorist picks what 
he takes to be a manageable subset of the word’s capa- 
cious denotation, with as little offence as possible to the 
word’s also large connotation. This is a perfectly rea- 
sonable thing to do and an instructive reminder of the 
rule not to bite off more than one can chew. Still, subset 
choices are at some risk of selection bias, which might, 
in turn, incline a theorist to presume that the theory of 
his own bite of the pie is in some ways canonical. 


2. Reasoning 


There is a question about arguing and reasoning. Are 
they One or Many? On the One view, arguing is just rea- 
soning out loud, whereas reasoning is just arguing with 
oneself privately. The Many view is supported by the 
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fact that young human beings routinely draw inferences 
before acquiring the linguistic means for making argu- 
ments. Reasoning is typically a search for conclusions 
from premisses at hand, whereas arguing is often a search 
for premisses to support some already hoped-for conclu- 
sion. Robert Pinto (1995) sees an argument as an invita- 
tion to draw an inference, that is, to adjust the invitee’s 
belief-state in accordance with how its premisses stand 
to its conclusion and he, the invitee, stands to its prem- 
isses. While Sue can invite Bill to draw the conclusion 
she sees as following from her proffered premisses, Bill is 
often able to draw his own inferences without invitation 
and also without the need of having to make the case 
to himself for drawing it. It would be an overreach to 
dismiss these solo inferences as inherently defective or 
subpar. Indeed, the frequency of argumentally uninvited 
inferences considerably exceeds the frequency of invited 
ones (Woods, 2013/2014). 

Another still unsettled question are the relations 
between good reasoning, good reason-giving, and the 
exercise of rationality. On one finding, a reasoner is rea- 
soning well when (or to extent that) he is reasoning in 
the way that a rational agent does (or would). Likewise, 
someone is good at giving good reasons for something 
when (or to the extent that) she does so in the way that 
a rational agent does (or would) in those same circum- 
stances. Especially pinching is the issue of how to under- 
stand situations in which a theory’s rules of rationality 
are systematically disobliged by the reasoning-behavior 
of actual human beings. Does massive discompliance 
discredit the theory, or can the theory save itself by 
invoking its normative authority over human behavior 
(see chapter 2.5 by Klauer, this handbook)? 

There are two different ways of approaching the nor- 
mativity question, not only in the theory of argument 
but in all the sciences of both empirically discernible 
and normatively assessable human behavior. Some hold 
that normative rules are what the theorists themselves 
take the common intuitive view of them to be (Johnson, 
2000). Others hold that rational performance is how the 
ideally rational agent would perform. Typical of these are 
Bayesian epistemologists, classical decision theorists, 
and neoclassical economists (see chapter 4.5 by Chater 
& Oaksford and chapter 9.4 by Dhami & al-Nowaihi, 
both in this handbook). Floating in these precincts are 
ideal norms to which no human being could approxi- 
mate in any finite degree: one provides that the ideally 
rational agent closes his beliefs under truth-preserving 
consequence. Another provides that when an ideally 
rational decision is taken, the decider has full command 
of all information of material significance. Neoclassical 
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economists postulate that an ideally rational economist 
is subject to the infinite divisibility of her subjective util- 
ities, largely because this enables economics to harness 
the mathematical firepower of the calculus. In each of 
these cases, it is put about that any real-life human agent 
is performing rationally to the extent that he approxi- 
mates to the ideal norms. So conceived of, the norms are 
accurate descriptions of how the ideally rational agent 
performs and an inaccurate description of how we our- 
selves perform. Even so, the story goes, the empirically 
false postulates are normatively binding upon us. This is 
not a well-received doctrine in by far most of the sec- 
tors of argumentation theory that are reviewed in this 
note. Of course, this is not to deny the approximable 
standards of goodness implicit in cognitive practice. For 
a good review, see the Introduction to this volume by 
Knauff and Spohn. 


3. Toulmin and the Rhetorical Turn 


In 1958, persuasion-arguments had a flourishing home 
in rhetoric but had largely disappeared from logic’s the- 
oretical halls.* This would start to change when Toul- 
min became aware of speech communication studies 
and informal logicians became aware of Toulmin.* In 
the textbooks of the time, natural-language arguments 
were mainly conceived of in the as-such way. Logic’s task 
was to provide mapping rules* from subsets of real-life 
arguments to structures of entities constructed from ele- 
ments of an uninterpreted formal language. Called “log- 
ical forms,” the idea was that if a property of interest for 
a natural-language argument—validity, say—could be 
shown to have a formal counterpart instantiated by the 
original argument’s logical form, then the formal prop- 
erty’s natural-language counterpart would also be instan- 
tiated there.° This is problematic: it is easily shown that 
there is no strict one-to-one correspondence between 
valid natural-language arguments and arguments hav- 
ing a valid logical form in some or other logistic sys- 
tem.’ Toulmin made a twofold departure in The Uses of 
Argument. He abandoned the mapping-to-logical-form 
test for properties of logical interest, and he abandoned 
the preoccupation with argument as-such and turned 
toward arguments in-use. 

Toulmin is best known for the model of argument 
that bears his name. Its core idea is that arguing is case- 
making and that cases are made by human beings for the 
entertainment and assessment of other human beings. 
When an argument is under way, its would-be conclu- 
sion is a “claim,” a proposition on behalf of which a 
case is being made and whose having been made is a 
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condition of its meriting acceptance. Claims (C) are sup- 
ported by “grounds” (G), which take the form of data, 
facts, or evidence. Grounds answer challenges claims. A 
“warrant” (W) is a statement that in turn endorses the 
support rendered by the ground. Claims, grounds, and 
warrants are necessary conditions for something’s being 
an argument. The next three parameters are discretion- 
ary, depending on context. A “backing” (B) lends sup- 
port to a warrant if it’s in doubt. A “rebuttal” (R) is a 
hedging device, indicating that in default of contrary 
indications, the backing is secure. A “qualifier” (Q) mod- 
ifies an arguer’s confidence at any given point. When 
these conditions are met, the first three always and the 
next three as may be, the arguer has made an argument 
for his claim. Toulmin’s own example of an argument 
that fits his model can be set out as follows: “Nigel is a 
British citizen” (C), “Nigel was born in Bermuda” (G), “A 
person born in Bermuda is a legal citizen of Britain” (W), 
“Nigel trained as a barrister in London, specializing in 
citizenship, so he knows that a person born in Bermuda 
is a legal citizen of Britain” (R). “Unless, that is to say, 
he has betrayed Britain and spied for another country.” 
Q is a discretionary prefix, attachable as context might 
indicate. 

It is puzzling why warrants don’t serve as premisses in 
the arguments they support. Toulmin concedes that Ber- 
mudians are British citizens and Nigel is a Bermudian. 
He allows that it follows that Nigel is a British citizen, 
but it follows from “Nigel was born in Bermuda” alone 
but not in the absence of the fact that Bermudans are 
British citizens. Although Toulmin didn’t adequately dig 
into what brings this to pass, he was on to something 
important. Not everything necessary for an argument’s 
logical success is suitable for use as a premiss (Carroll, 
1895; Ryle, 1950) or, as in some cases, even capable of 
being formulated as one.® The reason why is that back- 
ground information is often implicit and tacit (Polanyi, 
1966; Woods, 2021). 


4. Hamblin contra the Logicians 


Toulmin was disappointed in the quality of the introduc- 
tory logic textbooks at midcentury. Twelve years later, 
the Australian philosopher Charles Hamblin (1970) 
turned the disappointment into a scandale. He scorned 
the fallacy chapters of the leading texts of the day for 
their silly and unrealistic examples. The textbooks recy- 
cled what their predecessors had to say of the fallacies, 
as they in turn recycled what their own predecessors 
said, resulting in a dog’s breakfast “so incoherent that 
we have every reason to look for some enlightenment 
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at its historical source” (Hamblin, 1970, p. 50). Ham- 
blin follows his own advice with admirable effect. His 
book Fallacies traces the history of fallacy theory from 
its inception in antiquity, revealing its presence in 
every succeeding era until its post-Fregean fade-out in 
the last fifth of the 19th century. The survey shows the 
adaptability of Aristotelian themes to the freshly arising 
insights of his successors. Fallacies betrays no impatience 
with formal logic. Hamblin was drawn to a formalized 
dialectical model of argument and devoted two chap- 
ters to it. In so doing, he clearly caught a wave. Formal 
dialectic was on its way to being a growth industry (see, 
e.g., Henkin, 1961; Hintikka, 1968; Lorenz, 1961; Loren- 
zen, 1960; Lorenzen & Lorenz, 1978; Rescher, 1977). 

Fallacies had a second and more serious indictment to 
file: the reason there is no scholarship or depth in those 
pallid fallacy chapters is that logicians had betrayed their 
heritage. Aristotle invented their subject and placed the 
fallacies project at its dead center. But as Hamblin dole- 
fully notes, “We have no theory of fallacy at all, in the 
sense in which we have theories of correct reasoning 
or inference” (Hamblin, 1970, p. 11). Fallacies was a cri 
de coeur for logicians to reattach the fallacies project to 
logic. Unlike Toulmin, Hamblin wasn’t reaching out to 
the speech communication community. Hamblin was 
calling upon logicians to produce a thick theory of our 
mistaking something bad for something good, attended 
by the means to stop doing it. It is a given that an intel- 
lectual instrument purpose-built for the logicism of 
Frege, Whitehead, and Russell couldn’t be of much use 
in the analysis of paralogismoi.’ A logician whose only 
professional interest lay in the foundations of mathe- 
matics isn’t likely to have much working capital or appe- 
tite for fallacy theory. Hamblin is perfectly aware of this. 
His point is that securing the foundations of arithmetic 
shouldn’t be the sole business of logic. 

One of the merits of Hamblin’s indispensable book 
is the care with which it plots the historical course 
of fallacy theory, and thereby of logic itself, from the 
4th century B.c. to the present day. Its author also found 
space for a chapter on the Indian tradition in logic. A 
further virtue is the respect it shows medieval contribu- 
tions to logic, from which Hamblin draws considerable 
inspiration for his own advances in formal dialectic, in 
his chapter 8. As already mentioned, Hamblin is in the 
vanguard of a more general revival of formal dialogue 
and dialectical logic." 

It cannot be said that logicians flocked to Hamblin’s 
challenge and still less that he himself produced the the- 
ory he called for. I say this notwithstanding the right- 
ful interest that chapter 8 has attracted to the Hamblin 
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Game H. All the same, from that day to this, not a single 
article on fallacies has yet appeared in the Journal of Sym- 
bolic Logic. 


5. The Revival of Fallacy Theory 


In 1972, there appeared the first of a series of papers by 
John Woods and Douglas Walton on the fallacies that 
would extend to the mid-1980s. Woods was a philo- 
sophical logician working on the modal character of 
the entailment relation, and Walton, his former stu- 
dent at the University of Toronto, was an action theo- 
rist. In 1971, they were briefly in San Francisco, where 
they resolved to reserve the next two years to clear up 
the fallacies and then get back to their other business. 
Forty-seven years later, they are both still writing about 
fallacies. Woods and Walton accepted Hamblin’s insis- 
tence that fallacy theory is the proper business of logic. 
For each fallacy, they would seek an existing theoretical 
framework and adapt it to model the logical structure of 
the fallacy at hand. Theirs was a benign and not uncom- 
mon parasitism, mining the tried and true for some not 
originally intended good. Fallacies: Selected Papers 1972- 
1982 (Woods & Walton, 1989/2007) makes selective use 
of intuitionist, modal, and relatedness logics; causal and 
epistemic logics; and erotetic (interrogative), deontic, 
and dialectic logics. The book’s chapter 10 shows how 
a cumulative model of enquiry based on the semantics 
of Kripke’s intuitionist logic can block circular argu- 
ments.'’ Chapter 8 appropriates Burge’s (1977) aggregate 
theory for the analysis of the composition fallacy. Scat- 
tered throughout the book are adaptations of numerous 
frameworks for dialogue games, some of their own mak- 
ing.'* Overall, the Woods-Walton Approach (as it came 
to be called) embodies the maxim that if the foot of a fal- 
lacy fits the shoe of a suitably retrofitted logic, then the 
fallacy should wear it. It is a pluralistic approach eschew- 
ing any notion that a beautiful treatment of some one 
fallacy could be made canonical for how to treat them 
all. Some critics, notably Rob Grootendorst, protested 
that on the Woods—Walton Approach (WWA), every fal- 
lacy has its own logic. Just so! 

In chapter 1 of Woods and Walton’s Selected Papers, 
first published in 1972, it was shown that the truth 
conditions on the logical consequence relation cannot 
serve as reliable rules of deductive inference.”* A running 
theme of the WWA is that the informal fallacies, if fal- 
lacies they be, are not inherently fallacious in virtue of 
syntactic form. Whether an argument or piece of reason- 
ing is fallacious is always affected by contextual particu- 
larities, such as the goal-directedness of its parts. What 
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is true of the logics of the WWA is true of informal log- 
ics across the board: they are practically oriented logics. 
What the WWA also had, some of the others would reject 
as overconfidence in the assumption that philosophical 
treatments of the fallacies are best achieved under the 
gravitational pull of successful preexisting theories." 

The Woods-Walton papers were not by any means 
the only ones of note to appear in the close aftermath 
of 1970.'° For a while, the WWA dominated by force 
of numbers, if not always the press of merit. In time, it 
would be clear that the greater value of the WWA was the 
stimulus it provided for further work on fallacious rea- 
soning, some at rather substantial levels.'® Not the least 
of these developments are Walton’s 40 or so books either 
directly on the fallacies or significantly related to them. 
Since the early days, each of the WWA parties has parted 
company with their shared model, what with Walton’s 
circumspect expansions of the pragma-dialectic model 
(see below) and Woods’s move to a naturalized logic in 
company with a causal-response epistemological relia- 
bilism (Woods, 2013/2014; chapter 3.2 by O’Brien, this 
handbook). 


6. Informal Logic contra the Textbooks 


Toulmin didn’t like the textbooks of the 1950s, and Ham- 
blin abjured the ones of the 1970s. The two leading texts 
of those times were Irving Copi’s Introduction to Logic 
(1953) and An Introduction to Logic and Scientific Method, 
by Morris Cohen and Ernest Nagel (1934). Neither 
deserved the scorn that would be heaped upon them, 
notwithstanding the slightness of their fallacies cover- 
age. In 1971, something consequential happened: How- 
ard Kahane published Logic and Contemporary Rhetoric. 
Spurred by the student-power turbulence of the day, the 
standard textbooks were scorned for their irrelevance to 
the current upheavals. Kahane’s book reflected two facts 
welcomed by its new readers: one was that mainstream 
symbolic logic had strayed too far from the give-and-take 
of good and bad reasoning on the ground. The other was 
that rhetoricians had stayed closer to home and offered 
a broader, more varied, and more appealing assortment 
of argument strategies. The relevance revolution was 
not confined to introductory logic texts,'’ but it was the 
spur that motivated teachers of introductory courses to 
write new and more responsive ones. The now-spurned 
old primers would be placed on their Index Librorum Pro- 
hibitorum.'® Correspondingly, teachers of introductory 
logic courses were motivated to push back against what 
they rightly saw entirely as the expropriation of logic by 
mathematicians. The two developments were joined at 
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the hip, one to “relevantize” the textbooks, the other 
to “demathematicize” logic and restore it to its former 
primacy in philosophy. Jointly, the two developments 
marked the birth of late-20th-century informal logic. 

Beginning with their Logical Self-Defence in 1977, Ralph 
Johnson, J. Anthony Blair, and their colleagues established 
at the University of Windsor (Ontario) the organizational 
and communicational infrastructure for informal logic 
(IL). In so doing, they displayed an enduring steadfast- 
ness and remarkable networking skills. Windsor is home 
base for triennial meetings of the Ontario Society for the 
Study of Argumentation and also the place of publication 
of Informal Logic, the journal of record for that subject, 
and its related monograph series, Windsor Studies in Argu- 
mentation. Informal logicians have never seen fit to band 
together and issue manifestos. Windsor is head office 
infrastructurally but not doctrinally. Perhaps the better 
part of what explains this openness is that informal logi- 
cians tend rather more to agree on what they don’t like 
and rather less on what they do like. 

One point of near-universal agreement is IL’s hostil- 
ity to the suggestion conveyed by the textbooks of the 
day to the effect that first-order mathematical logic 
gives the best way to represent and evaluate deductively 
valid inferences as they occur in real life (Govier, 1987; 
Scriven, 1980). It is strange that on this very point, there 
is scant reference to Harman (1970). One can also see 
in the dissatisfactions of 1970 and onward a reaction to 
the entry of formal semantics, in the manner of Tarski, to 
the philosophy of language. Since language is the princi- 
pal medium of argument, it is only reasonable to expect 
that its philosophical theorists would have something 
insightful to say about it. To put it succinctly, informal 
logicians were resisting the idea that the best way to 
construct a philosophical theory of real-life argument 
is to craft it in the way that Tarski did for the natural- 
language truth-predicate in “The Concept of Truth in 
Formalized Languages” (1935/1983), on one reading of 
which the natural-language predicate “true” comes out 
as transfinitely ambiguous. In their determination not to 
produce this sort of semantics for the concept of argu- 
ment, informal logicians do themselves credit in respect- 
ing the realities of human speech. 

It is virtually impossible to produce a nontrivial over- 
view of the characteristic substance of informal logic, or 
in what its intellectual distinctiveness lies. Some writers 
concentrate on the premissory end of argument, others 
on the conclusional end, and others still on the premiss— 
conclusion link. Beyond the truisms, there is little by way 
of solid consensus. For example, Johnson’s and Blair’s 
criteria for argumental goodness—premiss-acceptability, 
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premiss—conclusion relevance, and sufficiency of prem- 
isses for conclusion-drawing—have had a long play. But, 
as they say, the devil is in the details, concerning which 
there are interesting differences of opinion:'” some 
informalists see themselves as normative theorists, while 
others are more interested in descriptive accuracy. Some 
think that informal logic won’t flourish unless it stays 
on top of developments in epistemology.” Others insist 
that the target of good argument is interparty concor- 
dance, not knowledge, even if truth might sometimes be 
a contingent collateral benefit of it.7’ Further unsettled 
questions are whether argument and inference are essen- 
tially the same, whether argumentation theories should 
be empirically sensitive, whether arguments are inher- 
ently (or even mainly) dialectical—that is, attack-and- 
defend contests—and whether courses in informal logic 
enhance one’s critical-thinking skills (Blair, 2019). Some 
emphasize the importance of taking metadialogues and 
meta-arguments seriously into account (Finocchiaro, 
2013; Krabbe, 2003). However, this is not to say that the 
informal logic idea is beyond informative articulation 
(see, e.g., Hitchcock, 2007; Johnson, 1996). 


7. Pragma-Dialectics 


If informal logic carries a nominal Canadian identity, 
pragma-dialectics (PD) has a more substantial Dutch one. 
It began with van Eemeren and Grootendorst (1984) 
as a simplified version of the sort of theory advanced 
in Barth and Krabbe (1982), a difficult and mathemati- 
cally sophisticated book entitled From Axiom to Dialogue. 
The pragma-dialectical book—Speech Acts in Argumenta- 
tive Discussion—treflected the authors’ confidence that 
conflict-resolution arguments could successfully be 
modeled without the encumbrance of complex math- 
ematicized formalisms. The counterpart city to Wind- 
sor is Amsterdam: it is the conference home of the 
International Society for the Study of Argumentation 
(ISSA) and of the journal of record Argumentation and its 
related monograph series. People who work in informal 
logic tend to have more arrows in their quiver than the 
Amsterdamers do. While many informal logicians write 
about other things,” pragma-dialectics is the sole preoc- 
cupation of those who espouse it. While it is true that 
there is no party line in Argumentation—the Amsterdam 
journal—the same is not true of its practitioners. I have 
sometimes heard it bruited in Amsterdam that there 
is a reason for this difference: the pragma-dialectical 
approach has produced a canonical theory, and the infor- 
mal logic approach has not. It takes at least two condi- 
tions for a theory to achieve canonical status: it must 
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stand tall in the marketplace of ideas, and it must have 
a large, durable, and approving readership. There are cer- 
tainly widely acclaimed individuals in the IL community, 
and some of their views have had more staying power 
than others. True, but the PD theory is a house-brand, and 
nothing in IL is quite that. 

The PD theory has two main parts: the first is a nor- 
mative theory of critical discussions furnished by an 
ideal model of them. The second part of the PD theory 
incorporates, under the heading “strategic maneuver- 
ing,” features derived from rhetoric and designed to 
make critical discussions persuasive. I’ll sketch the first 
and most central part of the theory, beginning with the 
10 critical discussion rules. The first thing to note is a 
change in the meaning of “fallacy.” A fallacy is now con- 
ceived of as any violation of the discussion rules. 

Paraphrasing van Eemeren, Grootendorst, and Snoeck 
Henkemans (2002), the rules can be paraphrased as 
follows: 


* Parties are free to advance standpoints and also to 
challenge them. 


* The burden of proof should lie on him who has 
advanced the standpoint. 


* Attack- and defend-arguments must be on topic. 


* Parties must not deny their own unexpressed prem- 
isses or misattribute an unexpressed premiss to the 
other. 


* Parties must not use premisses not accepted by the 
other party or deny a premiss accepted by their 
interlocutors. 


* The only conclusive defense of a challenged stand- 
point is by way of an appropriate argumentation 
scheme correctly applied. 


+ An argument is conclusive only if it is valid or can be 
made so by the explicitization of implicit premisses. 

* Ifa defense fails, it must be withdrawn, and if it suc- 
ceeds, the challenge must be retracted. 


* Moreover, parties must not speak in an unclear or 
confusing manner and must try to interpret one 
another fairly. 


The rules themselves jointly constitute a standpoint. It is 
disputed whether it satisfies its own rule 9. 

The PD approach has its critics, of course. Here are 
some of the main misgivings: (1) It is not clear whether 
the rules biconditionally define the PD’s own concept of 
“critical discussion.” (2) If they do, it is not clear whether, 
or to what extent, the PD model is actually instantiated 
in conversationally expressed differences of opinion in 
real life. (3) If the theory is empirically inadequate, it is 
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not clear whether it could be saved. (4) Could it be saved 
by its status in an ideal model? (5) Could it be saved by 
the ideal model’s normative authority? (6) The deductivist 
validity condition is problematic: any invalid argument 
can be validated by adding as a premiss a conditional 
whose antecedents are its other premisses and whose 
conclusion is its conclusion. (7) Finally, how well does 
the model “travel”? Concerning this last question, it is 
put about that the PD model is effective in the analysis 
and evaluation of collective-bargaining treaty negotia- 
tion, parliamentary debates, political arguments, argu- 
ments about health policy, and visual argumentation. 
Iam bound to say in reply that, however questions (1) to 
(6) are to be answered, in all good conscience the answer 
to (7) must be that it doesn’t travel especially well. 

Like all those in the reference class of this note, invo- 
cations of idealized normative models leave two critical 
chores undone: one is to specify the conditions under 
which an account of something of a kind K is a “model” 
of its behavioral instantiations. Second, what makes the 
model an “idealization” of K’s behavioral instantiations? 
Wherein lies its “normative authority” to bind in our 
own behavioral instantiations of K? For a thorough and 
carefully balanced view of these and related matters, one 
could do no better than consult Blair (2012, chapter 20). 

This leaves the question, Why does the PD approach 
have such appeal? A pair of answers come to mind. The 
less serious one is this: it is said in Windsor and other like 
places that the appeal of the PD approach lies in its easi- 
ness. The more serious answer is that the PD approach’s 
underlying pragma-dialectical character reflects some 
key truths about how human beings advance their views 
in reason-giving ways, for the consideration of other 
humans who think otherwise. If Blair is right, the diffi- 
culties with the PD approach might derive from its own 
theoretical handling of the underlying idea. If so, it could 
be that its supporters are rather more approving of the 
idea than of the theory. 


8. More Recent Developments 


Developments of importance include the expanding 
migration of argumentation theory to the analysis and 
evaluation of legal reasoning (Feteris, 1999; Prakken, 
1997; Sartor, 1995; Walton, 2002; Woods, 2015/2018; 
Zenker, Dahlman, Baath, & Sarwar, 2015), a renewed 
interest in argument by mathematical logicians (Bar- 
ringer, Gabbay, & Woods, 2012; Gabbay, 2013; van 
Benthem, 2011), an enlarging rapprochement between 
logic and psychology (Oaksford & Chater, 2007; Pa- 
glieri, Bonelli, & Felletti, 2016; chapter 5.5 by Hahn & 
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Collins and chapter 5.4 by Gazzo Castafieda & Knauff, 
both in this handbook), and the hook-up between the 
argumentation schemes literature, on the one hand, and 
software engineering and AI, on the other. We also see 
in this alliance an interesting overlap with legal argu- 
mentation (Verheij, 2003): initially introduced by Perel- 
man and Olbrechts-Tyteca (1958) and Hastings (1962), 
an argumentation scheme is a pattern of reasoning in 
everyday kinds of argument, adjoined to which are 
questions designed to facilitate the evaluation of argu- 
ments thus configured. For example, are the premisses 
accurate? Are there some unexpressed but load-bearing 
premisses? Might there be some allowable exceptions?” 
Under what assumption would the scheme in question 
intelligibly be construed as a successful form of argu- 
ment? Would a reasonable answer to that question make 
it defeasibly reasonable to regard the assumption as pro- 
visionally true?”* 

Argumentation Schemes for Presumptive Reasoning (Wal- 
ton, 1996) reveals a preoccupation with presumption. 
Walton tends to identify presumptive reasoning with 
defeasible reasoning. Some say that neither is, strictly 
speaking, a species of the genus “reasoning.” A piece of 
reasoning is defeasible just because it doesn’t close the 
world and therefore lies open to the possibility of defeat 
or weakening. Presumption—some say—is no kind of 
reasoning at all but rather a reasoner’s disposition to 
view defeating conditions as surprising. In many of these 
patterns of fallacious argument, the questions they raise 
are thought to be helpful in distinguishing the fallacious 
from the nonfallacious. For Walton (2013), “a fallacy is 
an argument, a pattern of argumentation, or something 
that purports to be an argument, that falls short of some 
standard of correctness as used in a conversational con- 
text and poses a serious obstacle to the realization of the 
goal of the dialogue” (p. 215). In essence, his is the PD 
conception. Walton’s further purpose was to show how 
argument schemes can help students in analyzing and 
judging arguments. In Walton (1996), there are 29 such 
schemes, but in Walton, Reed, and Macagno (2008), 
the list has swollen to 96, a daunting concordance for 
a student to keep on top of but no chore at all for a 
computer. It is ample evidence of Walton’s penchant for 
a pluralism that gives a multitude of different ways for 
something to be a fallacy, no fewer than the number of 
a scheme’s rules times 96. 

Computer modeling presents all theorists of empiri- 
cally discernible and normatively assessable behavior with 
interesting challenges. If the theorist in question seeks 
computer models of his account, he’ll have to formulate 
them in a way that enables the software engineer to get 


374 


hold of them. The argumentation schemes approach has 
had substantial appeal for AI engineers whose goal is to 
model them in ways that assist with “argument-mining” 
and multiagent analysis. This is brought about by feeding 
in answers to the questions raised by a scheme as further 
premisses of defeasible arguments now made valid by the 
additions. Notwithstanding the formidable reach of com- 
puter modeling, it is virtually impossible to bring off until 
data sets are suitably massaged for computational grasp. 
In so saying, we are back where we started. What wears 
the trousers here? The data or the mathematical models 
that seek them out? 

I close with a remark from Christopher Tindale’s 
Introduction to Blair (2012, p. xiv). Although specifi- 
cally about Walton, it applies equally to the argument 
schemes approach in general: 


In the review of Walton he asks some fundamental ques- 
tions of argumentation schemes: from where do they come, 
and from where do they derive their probative force? The 
origins of the Walton list had not been clear, and their 
cogency had been taken for granted by a number of theo- 
rists. But not by Blair. In similar fashion, he challenges the 
critical questions that accompany the schemes. What moti- 
vates these questions, and how do we know when a list is 
complete? 
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Notes 


1. A property is semi-decidable in a theory just in case there 
is a finitary algorithmic procedure for determining that some- 
thing has it if indeed it does have it. 


2. Roughly speaking a theory is of first order just in case its 
sole objects of reference are individuals. In a second-order 
theory, both individuals and properties or classes are objects 
of reference. 


3. A still-influential work of that same year is Perelman and 
Olbrechts-Tyteca (1958). 


4. Toulmin (1958), with Brockriede and Ehninger (1960) respond- 
ing and Toulmin, Rieke, and Janik (1979) responding in turn. 


5. Also mistakenly called “rules of translation.” 


6. Think also of entailment, deducibility, consistency, prov- 
ability, equivalence, and interdeducibility. 
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7. Consider, for example, the valid argument, “The shirt is red, 
so the shirt is colored.” There is no logistic system in which it 
maps to a valid form. 


8. For example, is memory or background information helpful, 
even when not decomposable into well-individuated proposi- 
tionally expressible units? 


9. Logicism is the doctrine according to which every theorem 
of arithmetic is provable from the laws of symbolic logic. 


10. In some of the more highly formalized ones, decidability 
and semi-decidability proofs are an intelligible, and sometimes 
even attainable, objective (van Benthem, 2011). 


11. Cumulative models preclude the retraction of answers in a 
question-and-answer game. The preclusion originated in Aris- 
totle’s treatment of refutation arguments. 


12. For example, Circle Games, DD games, and the so-called 
Woods-—Walton fragment. 


13. In medias res, they were made aware that the same point 
had been made in Harman (1970). 


14. See, for example, Rescher (1976). 


15. See, for example, Massey (1975), Broyles (1975), Barker 
(1976), and Mackenzie (1979). 


16. Later works of note include Walton (1984), Govier (1987), 
van Eemeren and Grootendorst (1992), Hansen and Pinto 
(1995), Johnson (2000), and Tindale (2007). See also the papers 
in Hitchcock (2017), Blair (2012), and Wohlrapp (2014). Also 
important is Plantin (2018). 


17. The student-power movement of the time rewrote univer- 
sity governance policies throughout the Western world. 


18. Oddly enough, the dismissal didn’t stop informal logicians, 
especially in the early decades, from mining the indexed books 
to motivate their theories. As one critic observed, “Pretty low- 
hanging fruit!” 


19. For premiss adequacy, see Freeman (2005). For relevance, 
see Gabbay and Woods (2003) and Walton (2004). For premiss 
sufficiency, see Woods (2013/2014, chapter 7). 


20. Plumping for the justified-true-belief model of knowledge 
are Biro and Siegel (1992) and Freeman (2005). Plumping for 
the causal-response model of reliabilism is Woods (2013/2014). 
See also chapter 3.2 by O’Brien (this handbook). 


21. See here van Eemeren and Grootendorst (2004). Essential 
counterreading is Blair (2012, chapter 20). 


22. For example, Finocchiaro on Galileo, Siegel on scientific 
relativism, Walton on courage, Barth on Quisling, Govier on 
peace, and Woods on bioethics. 


23. Schemes are prefigured as topica in Aristotle’s monograph 
of that same name. 


24. See also Hitchcock (2017, chapter 14), Rawman and Reed 
(2009), and Wyner (2016). 
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Section 6 Conditional and Counterfactual Reasoning 


6.1 Conditional and Counterfactual Logic 


William B. Starr 


Summary 


A logic aims to capture certain rational constraints on 
reasoning in a precise formal language that has a precise 
semantics. For conditionals, this aim has produced a vast 
and mathematically sophisticated literature that could 
easily be the topic of a whole volume of this size. In this 
chapter, the focus will be on covering the major logical 
analyses developed by philosophers, the key issues that 
motivate them, and their connection to views developed 
in artificial intelligence, linguistics, and psychology. 


1. Analyzing Conditionals 


Research on conditionals makes use of a few key terms 
and notation: 


Conditional A sentence of the form “If A then B,” and its 
variants. 


Antecedent The A component of a conditional. 

Consequent The B component of a conditional. 

Notation: “If A then B” is represented in logics as A > B, 
where A and B are logical representations of two sen- 
tences A and B. 


Conditionals are typically divided into two broad classes:' 


Indicative Conditionals For example, “If Maya sang, then 
Nelson danced.” 

Counterfactual Conditionals For example, “If Maya had 
sung, then Nelson would have danced.” 


A crucial dividing line between indicatives and counter- 
factuals is that counterfactuals can be used felicitously to 
talk about situations where the antecedent is contrary- 
to-fact (or thought to be false; Stalnaker, 1975; Veltman, 
1986). This contrast is evident in (1) and (2):? 


(1) Maya has definitely never sung. #If Maya sang, then 
Nelson danced. 


(2) Maya has definitely never sung. If Maya had sung, 
then Nelson would have danced. 


Corresponding indicative and counterfactual conditions 
can also differ in truth-value (D. K. Lewis, 1973, p. 3) or 
at least in their justification (Adams, 1975). Assuming 
Oswald was a lone shooter, the indicative (3) is straight- 
forwardly true and justified, while the counterfactual (4) 
is false, or at least unjustified: 


(3) If Oswald didn’t kill Kennedy, someone else did. 


(4) If Oswald hadn’t killed Kennedy, someone else 
would’ve. 


It is worth emphasizing that the term “counterfactual” 
should not be taken too literally, as it does not mean 
that a sentence of this form must have a contrary-to-fact 
antecedent (Anderson, 1951). For example, the counter- 
factual (5) is used as part of an argument that the ante- 
cedent is true: 


(5) If there had been intensive agriculture in the pre- 
Columbian Americas, the natural environment 
would have been impacted in specific ways. That is 
exactly what we find in many watersheds. 


A logic of conditionals typically aims to say which 
arguments involving A > B are deductively valid.’ 


Deductive Validity An argument with premises P;,..., 
P, and conclusion C is deductively valid just in case 
it is impossible for the premises to be true while the 
conclusion is false. Notation: Pi, ..., PnFC. 


However, as discussed in section 4, some instead follow 
Adams (1975) and focus on inductive support: whether 
the premises being true, or highly probable, makes the 
conclusion highly probable. Either of these approaches 
counts as pursuing a semantic approach to the logic of 
conditionals. The semantic approach is well suited to cap- 
turing the pervasive context sensitivity of conditional rea- 
soning, which will be a prominent theme of this chapter. 

Section 2 will outline the major challenges for a 
logic of conditionals and explain why classical truth- 
functional logic is not adequate. This will motivate sec- 
tion 3, which discusses three different analyses that draw 
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on tools from modal logic: strict-conditional analyses, 
similarity analyses, and restrictor analyses. This section 
will also outline how these analyses have been devel- 
oped using a different approach to semantics: dynamic 
semantics. In section 4, the chapter will turn to analyses 
that rely instead on the tools of probability theory.* 


2. Logic, Conditionals, and Context 


How can we systematically specify what the world must 
be like if a given conditional is true and thereby capture 
patterns of valid deductive arguments involving it? It 
turns out to be rather difficult to answer this question 
using the tools of classical logic. Seeing why will help 
identify the key challenges for a logic of conditionals. 

The logical semantics developed by Frege, Tarski, and 
Carnap provided useful analyses of English connectives 
like and, or, and not using the Boolean truth-functional 
connectives a, v, and 7. In truth-functional semantics 
(see chapter 3.1 by Steinberger, this handbook), the 
meaning of a sentence is identified with its truth-value 
True (1) or False (0), and the meaning of a connective 
is identified with a function from one or more truth- 
values to another, as depicted in table 6.1.1. The best 
truth-functional approximation of “If...then...” is 
the material conditional 5. ADB is false when A is true 
and B false, and it is true otherwise (making it equivalent 
to 7A v B). This analysis is employed in most introductory 
logic textbooks because it captures three key logical fea- 
tures of conditionals:* 


Modus Ponens A->B,A FB. (If a conditional and its 
antecedent are true, its consequent must be true.) 

No Affirming the Antecedent A > B £ A. (A conditional 
can be true even when its antecedent is not.) 

No Affirming the Consequent A > B ¥ B. (A conditional 
can be true even when its consequent is not.) 


The intuitive appeal of modus ponens is clear in 
(6a). The appeal of the inferences in (6b) and (6c) can 


Table 6.1.1 
Negation (=), conjunction (a), disjunction (v), and material con- 
ditional (>) as functions of True (1)/False (0). 


A B aA AnB AvB ADB 
ul 1 0) 1 1 1 
1 0) 0) 0) 1 0) 
0) it al 0) 1 1 
0) 0) 1 0) 0 il 
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be indirectly confirmed by showing that a conditional 
can be consistent with the negation of its antecedent or 
consequent. 


(6) a. If Maya sang, then Nelson danced. Maya did sing. 
Therefore, Nelson danced. 


b. If Maya sang, then Nelson danced. But Maya did 
not sing. 

c. If Maya sang, then Nelson danced. But Nelson 
didn’t dance. 


To see that modus ponens is valid for ADB, note that 
when ADB is true and A is true, B must be true—as rows 
1, 3, and 4 of table 6.1.1 show. Row 4 shows that it is 
possible for ADB to be true without either A or B being 
true. 

Despite having some attractive features, the mate- 
rial conditional analysis is widely taken to be incorrect.® 
There are particular problems for analyzing English con- 
ditionals as D. But there are also general problems with 
any truth-functional analysis. Let us consider the more 
general problems first. 

Many counterfactuals have false antecedents and con- 
sequents, but some are true and some false. For example, 
(7a) is false—given that Joplin’s “Mercedes Benz” was a 
critique of consumerism—and (7b) is true: 


(7) a. If Janis Joplin were alive today, she would drive a 
Mercedes Benz. 


b. If Janis Joplin were alive today, she would metab- 
olize food. 


This is not possible on a truth-functional analysis: the 
truth-values of antecedent and consequent determine a 
unique truth-value for the whole sentence. 

With a bit of care, a similar point can be made about 
indicatives: suppose a standard die has been tossed, but 
you do not know which side has landed face up. Then 
(8a) is intuitively true, while (8b) is false: 


(8) a. If the die came up 2, it came up even. 


b. If the die came up 1, it came up even. 


This intuition persists even when you get to see that 
the die came up 3: it would seem that (8a) is true and 
(8b) false even though both have a false antecedent 
and consequent. 

Another kind of problem for truth-functional analy- 
ses centers on the context sensitivity of conditionals. 
The basic observation is that the truth-value of a condi- 
tional can vary from one context of use to another, even 
when the truth-values of the antecedent and consequent 
stay the same across the two contexts. 
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This is clearest with counterfactuals. Quine (1982, 
chapter 3) voiced skepticism that any semantic analysis 
of counterfactuals was possible by highlighting puzzling 
pairs like (9): 


(9) a. If Caesar had been in charge [in Korea], he would 
have used the atom bomb. 
b. If Caesar had been in charge [in Korea], he would 
have used catapults. 


But D. K. Lewis (1973, p. 67) took these examples to show 
that the truth-conditions of counterfactuals are context 
sensitive. The antecedents and consequents of (9a) and 
(9b) are all false, but in some conversational contexts, 
(9a) seems true and (9b) false, and in other conversa- 
tional contexts, (9a) seems false and (9b) true. Consider 
evaluating (9a) and (9b) in a context where we have 
explicitly discussed and agreed that Caesar was, first and 
foremost, a ruthless military leader. In such a context, 
(9a) seems like a true thing to say, while (9b) seems false. 
By contrast, consider a context where we have explicitly 
discussed and agreed that Caesar was, first and foremost, 
a technologically conservative military leader. Then (9b) 
seems like a true thing to say, while (9a) seems false. 

Similar examples illustrate the same point for indica- 
tive conditionals. Suppose we have mutually established 
that a die in our possession has 3 on every side except for 
one, which has 2. The die has been tossed, but we do not 
know how it came up. Then (10) seems true: 


(10) If the die came up even, it came up 2. 


Suppose we then find out that the die came up 3. It seems 
that (10) was still a true thing to say. However, consider 
a more ordinary context where we have mutually estab- 
lished that our die is a standard one. The die has been 
tossed, but we do not know how it came up. It would be 
false to say (10). Now suppose we later find out that the 
die came up 3. It still seems like (10) is false. This shows 
that the truth-value of (10) can vary from context to con- 
text, even when the truth-values of its antecedent and 
consequent are held fixed. 

These general problems for a truth-functional analysis 
are compounded by particular weaknesses in the material 
conditional analysis. This is particularly clear for counter- 
factuals. As rows 3 and 4 of table 6.1.1 make clear, ADB 
is true any time A is false. This means that the material 
conditional validates the following logical principle: 


Material Antecedent =A F ADB. 


But this means that all genuinely contrary-to-fact con- 
ditionals are true on a material conditional analysis. So 
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not only is (11a) incorrectly predicted to be true, but 
both (11a) and (11b) are predicted to be true despite the 
fact that they seem to be contradictory: 


(11) a. If the Earth hadn’t existed, the moon would 
have existed. 

b. If the Earth hadn’t existed, the moon wouldn’t 
have existed. 


For indicatives, this problem surfaces in another way: 
recall that indicatives are not felicitous to use when their 
antecedent has been explicitly denied. This is quite puz- 
zling if that pattern of use is actually a valid form of 
argument. 

Another major problem for the material conditional 
analysis stems from the fact that it validates 


Material Negation —(ADB) FA. 


But neither of the following are compelling arguments 
for the existence of God, despite having (plausibly) true 
premises: 


(12) a. It’s not true that if God exists, he’s a turnip. 
Therefore, God exists. 
b. It’s not true that if God had existed, he would be 
a turnip. Therefore, God exists. 


But the shortcomings of this analysis are instructive, as they 
establish some clear criteria for a more successful analysis: 


Non-Truth-Functionality The truth-value of a condi- 
tional is not determined by the truth-values of its 
antecedent and consequent. 

Context Dependence The truth-value of a conditional 
depends on certain features of the context in which 
it is used. 

Logical Constraints Conditionals do not obey Material 
Antecedent or Material Negation. 


This is the starting point for analyses that appeal to pos- 
sible worlds. 


3. Conditionals and Possible Worlds 


Table 6.1.1 helps make salient a crucial assumption of 
truth-functional semantics: the truth-value of a complex 
sentence is determined only by the truth-values of its 
component sentences in that row. Once the truth-values 
of A and B have been settled, the truth-value of A ~ B has 
been settled. But table 6.1.1 also makes salient an alterna- 
tive approach: what if the truth-value of a complex sen- 
tence is determined by the distribution of its component 
sentences’ truth-values across a number of rows? 
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Pursuing this alternative analysis requires clarify- 
ing exactly what “a row” is and settling on a particu- 
lar account of which distributions matter. Following 
the tradition in modal logic, one can think of the rows 
as possible worlds (Kripke, 1963). The two basic analy- 
ses surveyed in this section—strict-conditional analyses 
(“strict analyses”; section 3.1) and similarity analyses 
(section 3.2)—amount to different views about which 
distributions of truth-values across worlds make condi- 
tionals true. Sections 3.3 and 3.4 discuss two resources 
for extending these two kinds of analyses. Before explor- 
ing the analyses in detail, it is useful to state their shared 
assumptions and their main difference. 

Strict and similarity analyses both employ the idea of 
possible worlds w, which are simply ways the world could 
be or could have been. They are treated as primitive 
points in the set of all possible worlds W. They play a 
crucial role in assigning truth-values to sentences: a sen- 
tence A is only true given a possible world w, but since 
w is genuinely possible, it cannot be the case that both 
A and “A are true at w. Consider, then, a language with 
just three atomic sentences A, B, C, namely, “Maya ate 
apples,” “Maya ate bananas,” and “Maya ate cherries.” 
At least eight possible worlds are then needed, as listed 
in table 6.1.2:’ strict and similarity analyses both aim to 
say when a conditional like A > B is true in a given world 
w and do so on the basis of whether certain A-worlds 
relevant to w are also B-worlds. But they differ in terms of 
how they determine this set of worlds relevant to w. 

Different methods for selecting relevant worlds pro- 
duce a distinctive logical difference between strict and 
similarity analyses. This logical difference is easiest to 
illustrate with the following principle: 


Antecedent Strengthening AC F (An B)>C. 


Goodman (1947) observes that this does not hold for 
counterfactuals, since (13a) is true and (13b) false: 


Table 6.1.2 
Possible worlds for A, B, C 


A B C 
Wi 1 1 1 
W2 1 1 (0) 
W3 1 0 1 
Ws 1 0 (0) 
Ws (0) 1 1 
We (0) 1 (0) 
W7 (0) (0) d 
Ws 0) (0) 0 
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(13) a. If I had struck this match, it would have lit. 
S>L 


b. If I had struck this match and done so in a room 
without oxygen, it would have lit. 


(Sa 730) >L 


D. K. Lewis (1973, p. 10) dramatized this counterexam- 
ple by considering “Sobel sequences” such as (14). Sur- 
prisingly, adding more information to the antecedent 
repeatedly flips the truth-value of the counterfactual. 


(14) a. If I had shirked my duty, no harm would have 

ensued. 
I>-7H 

b. Though, if I had shirked my duty and you had 
too, harm would have ensued. 
UaU)>H 

c. Yet, if I had shirked my duty, you had shirked 
your duty, and a third person had done more 
than their duty, then no harm would have 
ensued. 


InUnT)>7H 


For indicatives, (15a) may be true when (15b) is not: 


(15) a. If Maya sang at the party, then Nelson danced 
at the party. 


b. If Maya sang at the party and Nelson wasn’t 
there, then Nelson danced at the party. 


Similarity analyses predict that antecedent strengthen- 
ing is invalid while strict analyses predict that it is valid. 
Strict analyses address the counterexamples above prag- 
matically: the conclusions of the arguments are false, but 
that is because pragmatic mechanisms force the conclu- 
sion to be interpreted relative to a different set of worlds 
than the premises. In doing so, these analyses appeal to a 
feature already highlighted above: conditionals’ context 
sensitivity. 

The general pattern, of which antecedent strengthen- 
ing is an instance, is: 


Antecedent Monotonicity If B F A then A>C FBC. 
Another instance of this is: 


Simplification of Disjunctive Antecedents (SDA) (A v B) = 
CF (A>C)a(B>C). 


Antecedent monotonicity also leads (indirectly) to (Starr, 
2019, section 2.1): 


Transitivity A> B, B>CFAC. 
Contraposition A — BF ~B—-W\A. 
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Counterexamples similar to those above have been 
presented by similarity theorists to transitivity, contra- 
position, and SDA (see, e.g., D. K. Lewis, 1973; McKay 
& van Inwagen, 1977; Stalnaker, 1968). Strict theorists 
have also aimed to address these examples pragmatically 
(e.g., Warmbrod, 1981). So the key issue dividing strict 
and similarity issues is the following: 


Pragmatic or Semantic Nonmonotonicity? Is the non- 
monotonicity of conditional antecedents best explained 
semantically or pragmatically? 


* Strict Theorists: pragmatically (e.g., Gillies, 2007, 
2009; Warmbrod, 1981) 

¢ Similarity Theorists: semantically (e.g., D. K. Lewis, 
1973; Stalnaker, 1968) 


This chapter will not weigh in on this question, instead 
presenting the basics of these two approaches. Anteced- 
ent strengthening will be the focus for concreteness, but 
it is just a representative of antecedent monotonicity 
more generally. 


3.1. Strict-Conditional Analyses 

Strict-conditional analyses began with the basic idea that 
A-B is true just in case all of the A-worlds are B-worlds 
(Peirce, 1896). This has been refined using a crucial tool 
from the semantics of modal logic: accessibility functions 
R. The function R takes a world w and returns the set 
R(w) of worlds that are accessible from, or relevant to, w 
(Kripke, 1963).° 


Strict Conditional A= B is true in w, given R, just in case 
every A-world in R(w) is a B-world. 


To illustrate this definition, consider a particular world 
w, and a particular accessibility function R; where R;(w7) 
corresponds to the set of worlds bolded in table 6.1.3.4 => B 
comes out true in this case: every bold world where A is 1 
is a world where C is 1. A=>C would come out false on a 
slightly different accessibility function R2, which is just like 
R; except it includes wz: a world where A is 1 and C is 0. R2 
in fact shows why antecedent negation does not hold for 
the strict conditional: “A is true in w7 and yet A= Cis false 
in wz, relative to Ro. Similarly, this is also a counterexample 
to material negation since -(A=C) is true in w,, relative 
to R2, and A is false in w7. These count as counterexamples 
given the following definition of validity. 


Modal Validity P;, ..., P, = C just in case for every w and 
R, if P;,..., P, are true in w, relative to R, then C is 
true in w, relative to R. 


According to this definition, antecedent strengthen- 
ing is valid, an issue to which we will return. Modus 
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Table 6.1.3 
Possible worlds for A, B, C; worlds in Ri(w7) in bold 

A B C 
Wi 1 1 1 
W2 1 1 (0) 
w; 1 0 1 
W4 1 (0) (0) 
Ws 0 1 1 
Wo 0 1 (0) 
W, 0 0 1 
We 0 0 0 


ponens is valid for the strict conditional if one requires 
that accessibility functions satisfy reflexivity: for all w, w is 
in R(w). Accordingly, the strict conditional addresses the 
main logical problems faced by the material conditional 
analysis. As discussed at the outset of this section, the 
strict analysis also captures non-truth-functionality: the 
truth-value of A=>B depends not just on the actual truth- 
values of A and B but on their truth-values across all the 
worlds in R(w). 

The context sensitivity of conditionals is central to con- 
temporary strict analyses, particularly when responding to 
examples like (13), (14), and (15). These examples seem to 
show that antecedent strengthening is not valid for condi- 
tionals, contrary to the strict analysis. While early work on 
the logic of strict conditionals did not include this compo- 
nent, subsequent work like Warmbrod (1981) has done so 
by treating R(w) as one aspect of context that changes as 
utterances take place. 

Warmbrod (1981) proposes that the accessibility func- 
tion R(w) corresponds to the background suppositions of 
the conversationalists and will change as the conversa- 
tion unfolds. In particular, Warmbrod (1981) proposes 
that R(w) also changes as the result of conditionals being 
asserted. Warmbrod (1981) notes that trivial strict condi- 
tionals are not pragmatically useful in conversation. A 
strict conditional A= C is trivial just in case A is incon- 
sistent with R(w)—there is no A-world in R(w). Asserting 
A=C in such a context does not provide any informa- 
tion: every conditional of the form A= X is true. But 
Warmbrod (1981) proposes that conversationalists adopt 
a pragmatic rule of charitable interpretation to make 
sense of why the speaker asserted A = C rather than a dif- 
ferent conditional: 


(P) If the antecedent A of a conditional is itself consis- 
tent, then there must be at least one A-world in R(w). 


Given this, R(w) can change as a result of asserting a con- 
ditional. This part of the view is central to explaining 
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away counterexamples to antecedent monotonic validi- 
ties, like antecedent strengthening. 

Consider again this counterexample to antecedent 
strengthening: 


(13) a. If I had struck this match, it would have lit. 


b. If I had struck this match and done so in a room 
without oxygen, it would have lit. 


On a strict analysis, if (13a) is true in a world w, rela- 
tive to Ro, then Ro(w) must exclude any worlds where 
the match is struck but there is no oxygen in the room. 
However, if (13b) is interpreted against Ro, the anteced- 
ent will be inconsistent with Ro(w) and so express a trivial 
strict conditional. According to Warmbrod (1981), inter- 
preting (13b) according to (P) forces the conversational- 
ist to adopt a new, modified accessibility function R;(w) 
where the presence of oxygen is no longer assumed. If 
this is right, then (13) is not really a counterexample to 
the validity of antecedent strengthening: the premise is 
true relative to Ro, and the conclusion is only false rela- 
tive to R;. Warmbrod (1981) and others develop versions 
of this account to address all of the proposed counter- 
examples to antecedent-monotonic patterns.’ These 
accounts do have an important limitation: they do not 
capture nested conditionals and do not actually predict 
how R(w) evolves to satisfy (P). Von Fintel (2001), Gil- 
lies (2007, 2009), and Willer (2017) offer accounts that 
remove these limitations, using the tools of dynamic 
semantics covered in section 3.4. 


3.2 Similarity Analyses 

The basic idea of similarity analyses is that A > C is true 
in w when C is true in all of the A-worlds most simi- 
lar to w (D. K. Lewis, 1973; Stalnaker, 1968; Stalnaker & 
Thomason, 1970). One way of precisely formulating this 
appeals to a selection function f, which takes a world w 
and a proposition p and returns the set of p-worlds most 
similar to w: f(w, p).'° This is used to define the truth- 
conditions of a similarity-based conditional, notated 


“ 


>,” as follows:" 


Similarity Analysis A > C is true in w, relative to f, just in 
case every world in f(w, A) is a C-world. 


While a strict analysis assumes a single set of relevant 
worlds R(w) for all conditionals, fcan select a different set 
of worlds for each different antecedent. For example, it is 
perfectly possible for f(w, A a B) to contain worlds that are 
not in f(w, A), even though all A ~ B-worlds are A-worlds. 
Different similarity analyses propose different constraints 
on f. The candidate constraints are given in table 6.1.4." 
To make clear that this permits f(w,A~B) to contain 
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worlds not in f(w, A) and how this invalidates antecedent 
strengthening, let’s consider a concrete example. 

Consider the worlds and selection function f, in 
table 6.1.5. A> C is true in We, relative to fi, since A and 
C are true in w3. But (An B) > C is false in we, relative to 
fi, since AaB is true and C false in wy». It is easily veri- 
fied that f, satisfies all four constraints in table 6.1.4. 
This should highlight the key difference between strict 
and similarity analyses: strict analyses assume a fixed set 
of relevant worlds for all antecedents, while similarity 
analyses allow the set of relevant worlds to vary from 
antecedent to antecedent—even among logically related 
antecedents. It is also worth highlighting a point that 
will matter later: nothing in the formal analysis requires 
f(w, p) to hold fixed facts of w, even if they are unrelated 
to p. This is clear with fi(A, we) = {w3}, which does not pre- 
serve the fact that B is true in we and that B’s truth may 
be unrelated to A being false in we. 

The constraints on selection functions listed in 
table 6.1.5 are partly motivated by our intuitive concept 
of similarity but also by logical considerations. Success 
enforces that the set of most similar p-worlds to w are 
in fact p-worlds. But the other constraints correspond to 
certain logical validities—see Starr (2019) for a thorough 
discussion of this. For the purposes of this chapter, only 


Table 6.1.4 
Candidate constraints on selection functions; p, q@QW and 
wEew 


success 


(a) F(w,p) Sp 


(b) f(w, p) = {w}, if wep strong centering 


(c) if fw, p) Gg and f(w,q) Gp then uniformity 
fw, p) = fw, q) 
(d) f(w, p) contains at most one world uniqueness 


Table 6.1.5 
Possible worlds for A, B, and C; fi(A,we) in bold, fi(A a B, We) 
underlined 


A B C 
Wi 1 1 1 
W2 I 1 0) 
Ww 1 0 1 
W4 1 0 0 
Ws (0) 1 1 
Wo (0) 1 (0) 
W7 0 0 1 
Ws 0) 0) 0) 
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strong centering will matter, as it ensures that similarity 
conditionals validate modus ponens." 

Material antecedent and material negation are invalid 
for the similarity conditional for the same reasons that 
they were invalid for the strict conditional. The falsity 
of A in w does not ensure that the A-worlds most similar 
to w are B-worlds. So =A KF A> B. A >B can be false in w 
when A is false in w, but in one of the A-worlds most 
similar to w, B is false. So -(A > B) F A. 

The context sensitivity of conditionals can be captured 
on similarity analyses by highlighting the fact that judg- 
ments of similarity are themselves context dependent 
(D. K. Lewis, 1973). As D. K. Lewis (1973, p. 67) details, 
different contexts can make salient different properties 
of the things we are talking about, and this affects what 
counts as a similar world to our own. This is illustrated 
with the pair in (9), discussed back in section 2: 


(9) a. If Caesar had been in charge [in Korea], he would 
have used the atom bomb. 
b. If Caesar had been in charge [in Korea], he would 
have used catapults. 


Consider a context where Caesar’s brutality is made 
salient. It will then be held fixed when determining 
which worlds where Caesar was in charge in Korea count 
as most similar to our own. As a result, (9a) will come out 
true and (9b) false. Other contexts will have the opposite 
effect. Quine’s (1982, chapter 3) claim that there is no 
fact of the matter whether these counterfactuals are true 
comes from failing to embed them in natural contexts. 
Subsequent work such as K. S. Lewis (2018) and Ippolito 
(2016) develops this idea. 

Many similarity theorists explicitly limited the analysis 
to counterfactuals (e.g., D. K. Lewis, 1973). But Stalnaker 
(1975) applied the analysis to indicative conditionals by 
saying that indicatives and counterfactuals differ in how 
they are context sensitive. Recall that indicatives and 
counterfactuals can differ in truth-value. Setting aside 
conspiracy theories, (3) is true and (4) false: 


(3) If Oswald didn’t kill Kennedy, someone else did. 


(4) If Oswald hadn’t killed Kennedy, someone else 
would’ve. 


Stalnaker (1975) explains this contrast in terms of a 
general account of how assertion works. The mutual 
assumptions of the conversationalists can be mod- 
eled as a set of worlds c called the context set—the set 
of worlds compatible with what everyone is assuming 
and with what everyone is assuming that everyone is 
assuming and so on. When a proposition p is asserted 
and accepted, then the new context set is cM p—the set 
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of all worlds in the previous context set and compatible 
with p. 


Stalnaker (1975) Analysis of Indicatives For an indicative 
conditional A > B, if w &c, then f(w, A) Ec. 


When evaluated in a world compatible with the context set, 
the most similar antecedent worlds must also be within the 
context set. 


This predicts (3) to be true, since it requires that its inter- 
pretation hold fixed the fact that Kennedy was killed— 
that proposition is part of the context set against which 
(3) is asserted. By contrast, (4) does not require that to 
be held fixed and so allows the most similar antecedent 
worlds to be ones where Kennedy was not killed at all. 
On this analysis, all conditionals are context sensitive, 
but indicative conditionals are specifically sensitive to 
the context set. This goes some way in explaining the 
observation that indicative conditionals are only felici- 
tous when their antecedent has not been explicitly ruled 
out—recall (1) and (2) from section 1. 


3.3. Restrictor Analyses 

The restrictor analysis of conditionals originates with 
D. K. Lewis (1975) and Kratzer (1981, 1986) and argues 
for a dramatic change in the logical analysis of condi- 
tionals. It has been assumed that conditionals have a 
logical form like A—B and that their analysis must pro- 
ceed by finding the right meaning for —. But restrictor 
analyses contend that this is wrong. An analysis of con- 
ditionals should begin with modal adverbs like must, 
would, might, and probably that occur in what is normally 
thought to be the consequents of conditionals. A con- 
ditional like “If Maya sang, Nelson probably danced” 
should be thought of primarily as a sentence of the form 
Probably(D). All the if-clause does is restrict the domain 
of worlds over which probably quantifies. While “Nelson 
probably danced” says that Nelson danced in most of 
the worlds, “If Maya sang, Nelson probably danced” says 
that Nelson danced in most of the worlds where Maya 
sang. 

D. K. Lewis (1975) and Kratzer (1986) argue for a 
restrictor analysis by observing that no uniform contri- 
bution can be assigned to AB that captures the dif- 
ferent meanings that conditionals have when different 
modal adverbs occur in the consequent. This argument 
will not be summarized here because restrictor analy- 
ses are not actually competitors to strict and similarity 
analyses—they simply specify a different form those two 
analyses can take: as Kratzer (1991, p. 649) details, mate- 
rial conditionals, strict conditionals, and similarity con- 
ditionals can all be modeled within a restrictor analysis. 
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A restrictor analysis provides a resource for extending the 
empirical coverage of strict and similarity analyses: while 
the conditionals considered so far involve a necessity 
modal of some sort, a more general analysis is needed.'* 
Further, a restrictor analysis provides a more flexible the- 
ory that could, in principle, provide an account on which 
conditionals are sometimes strict and sometimes similar- 
ity conditionals. For more on this approach, see Kratzer 
(2012). 


3.4 Dynamic Analyses 

This chapter has assumed that the meaning of a sentence 
corresponds to its truth-conditions: the set of worlds in 
which it is true. Formally, this means that the semantics 
is specified as a function [A] that maps sentences A of the 
formal language to subsets of W (the set of all possible 
worlds). Logic followed suit: it requires that valid argu- 
ments preserve truth. But there is another dynamic per- 
spective on meaning: it is the characteristic way a sentence 
changes a context in which it is uttered (Groenendijk & 
Stokhof, 1991; Heim, 1982; Kamp, 1981; Veltman, 1996). 

On Veltman’s (1996) approach, a dynamic semantics 
is specified as a function [A] that maps one information 
state s (a subset of W) to another information state s’. 
This is written s[A] =s', and it is said that s’ is the result 
of updating s with A. The meaning of A resides in the 
difference between the two states: one prior to the use 
of A and one posterior to the use of A. A expresses truth- 
conditional information about worlds by eliminating 
worlds from s. But it can also express a global property 
of the information s that is quite different than distin- 
guishing between worlds. 

For example, the sentence Might(A) tests whether s 
is consistent with A. If not, s’ is reduced to @. But if it 
is, then s’ is left as it is. On this view, Might(A) expresses 
a property of the information state s, without treating 
that property as something that distinguishes worlds in s 
from each other. Logic follows suit: valid arguments pre- 
serve information rather than truth. More specifically, 
updating with the conclusion after updating with the 
premises provides no additional information: 


Dynamic Validity P;,..., P, F C just in case for any infor- 
mation state s, s[P:]. . . [Pn] = s[Pi]. . . [Pnl[C]. 


Both strict and similarity analyses have drawn inspira- 
tion from this dynamic approach. 

Von Fintel (2001) and Gillies (2007) develop dynamic 
strict analyses of counterfactuals and argue that they 
can better explain ordering effects. Among them are 
reverse Sobel sequences, which are simply the reversal of 
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the sequences like (14) presented by D. K. Lewis (1973, 
p. 10). The important observation is that reversing these 
sequences is not felicitous: 


(14) a. If [had shirked my duty and you had too, harm 
would have ensued. 
b. #If I had shirked my duty, no harm would have 
ensued. 


Von Fintel (2001) and Gillies (2007) observe that similar- 
ity analyses render sequences like (14) semantically con- 
sistent. Their theories predict this infelicity by providing 
a systematic theory of how counterfactuals update con- 
text.'* These analyses involve richer models of context 
than Veltman (1996). 

Gillies (2004, 2009) develops a dynamic strict analy- 
sis of indicative conditionals on which s[A —B] tests 
that all the A-worlds in s are C-worlds. If the test is 
passed, s stays as it is. If it is failed, s is reduced to ©. 
This analysis, as well as the dynamic definition of valid- 
ity, navigates a number of tricky issues in conditional 
logic. Gillies (2004) uses it to diffuse counterexamples 
to modus ponens (McGee, 1985). Willer (2012) extends 
this solution to counterexamples to modus ponens 
presented by Kolodny and MacFarlane (2010). Stojni¢ 
(2016) integrates a dynamic strict analysis with a theory 
of discourse coherence and modal anaphora to address 
counterexamples to modus ponens and modus tollens. 
Gillies (2009) uses a dynamic strict analysis to counter a 
number of arguments (e.g., Edgington, 1995; Gibbard, 
1981), which say that it is not possible to assign truth- 
conditions to indicative conditionals that are stronger 
than a material conditional and weaker than a classical 
strict conditional and capture the import—export equiva- 
lence between A — (BC) and (Aa B) ~C. Nondynamic 
similarity theories invalidate import-export but offer no 
evidence in its favor. 

Starr (2014) proposes that the general meaning of 
conditionals is that given by Gillies (2009) but that 
counterfactuals contain an operator in their antecedent 
that allows them to talk about the most similar ante- 
cedent worlds. This operator is argued for on the basis 
of work in linguistics such as Jatridou (2000). Starr 
(2014) argues that this dynamic strict analysis of indica- 
tives and dynamic similarity analysis of counterfactuals 
solves a number of outstanding problems. Willer (2017) 
has developed a related account and argues that only a 
dynamic strict analysis can explain why indicative ver- 
sions of (14) are felicitous. This remains an active area 
of research. 
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3.5 Discussion 

Both strict and similarity analyses can overcome the 
basic shortcomings of the material conditional analysis. 
They validate modus ponens while invalidating mate- 
rial antecedent and material negation, and they are not 
truth-functional. They also leave room for the context 
sensitivity of conditionals. The main debate between 
them is whether nonmonotonic inferences like anteced- 
ent strengthening should be rendered invalid (similar- 
ity analyses) or whether the proposed counterexamples 
involve changes in context (strict analyses). This is a 
subtle and ongoing debate. 

There is, however, a more pressing challenge to both 
of these analyses. To explain how a given conditional 
like (17) expresses a true proposition, a similarity analy- 
sis must specify which particular conception of similar- 
ity informs it. 


(17) If my computer were off, the screen would be blank. 


Of course, the strict analysis is in the same position. It 
cannot predict the truth of (17) without specifying a par- 
ticular accessibility relation. In turn, the same question 
arises: on what basis do ordinary speakers determine 
some worlds to be accessible and others not? Theories 
like those discussed above do not directly address this 
question, as they are primarily concerned with the logic 
of conditionals. But a wealth of examples illustrates that 
there are systematic generalizations about how sen- 
tences are judged to be true, and it is not clear that strict 
or similarity analyses are well positioned to capture this. 
Readers interested in this next turn in the debate are 
directed to Starr (2019, section 2.5). 


4. Conditionals and Probability 


Conditional reasoning involves uncertainty and evaluat- 
ing the consequences of the world being different than it 
actually is. The study of probability has provided a suite 
of tools designed for just these purposes. It is no accident, 
then, that many philosophers, psychologists, and com- 
puter scientists have drawn on these tools to analyze the 
meaning and logic of conditionals. The seminal proposal 
here comes from Adams (1975), which analyzes condi- 
tionals in terms of conditional probability. This basic 
idea, as well as its development by philosophers (Edg- 
ington, 1995) and psychologists (Evans & Over, 2004), 
is surveyed in section 4.1. Recently, research in artificial 
intelligence (AI), philosophy, psychology, and linguistics 
has appealed to a related probabilistic tool: Bayesian net- 
works. Section 4.2 will survey those accounts. 
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4.1 Conditional Probability 
The seminal proposal from Adams (1975) is this: 


Adams’ Thesis The assertability of “B if A” is proportional 
to P(B | A), where P is a probability function represent- 
ing the agent’s subjective credences. 


Probabilities are real numbers between 0 and 1 assigned 
to propositional variables A, B, C, ... Adams takes these 
probabilities to reflect an agent’s subjective credence; for 
example, P(A) = 0.6 reflects that they think A is slightly 
more likely than not to be true.’® P(B| A) is the credence 
in B conditional on A being true and is defined as 
follows: 
P(AaB) 

P(B) 
Surprisingly, the logic of conditionals based on condi- 
tional probability developed by Adams (1975) is more 
or less the same as the similarity analyses pursued by 
Stalnaker (1968) and D. K. Lewis (1973). In particular, it 
invalidates antecedent-monotonic patterns, along with 
material negation and material antecedent. Its major dif- 
ference from similarity analyses comes in its account of 
context sensitivity and non-truth-functionality. 

This can be illustrated with examples discussed 
earlier. Recall the context of (8). You know a standard 
die has been tossed, but you do not know which side has 
landed face up. (8a) seems like a good assertion to make, 
while (8b) does not: 


Conditional Probability P(B|A) = 


(8) a. If the die came up 2, it came up even. 
b. If the die came up 1, it came up even. 


When you get to see that the die came up 3, it still seems 
right to say that (8a) was a justified assertion even if you 
would not now assert that conditional.'’ The same goes 
for (8b). Both intuitions are captured on a conditional 
probability analysis. The conditional probability of the 
die coming up even given that it came up 2 is 1, while the 
conditional probability of the die coming up even given 
that it came up 1 is 0. (8) was an example of non-truth- 
functionality, and the conditional probability analysis 
captures this: instead of the truth of a conditional being 
determined by the truth of its parts, the probability of 
a conditional is being determined by the probability of 
its parts. As it turns out, this view is even more radical. 
A conditional probability P(B|A) cannot be modeled as 
the probability that a conditional proposition B| A is true 
(D. K. Lewis, 1976). So P(B|A) should not be thought of 
as the probability that a proposition is true. For this rea- 
son, many philosophers articulate a conditional probabil- 
ity analysis as holding that conditionals do not express 
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truth-evaluable propositions but merely a ratio of cre- 
dences in truth-evaluable propositions (Edgington, 1995). 

The context sensitivity of conditionals is reflected in 
the core idea of the conditional probability analysis: con- 
ditionals express credences. As those credences change, 
so will the ratios between them. Consider again (10) and 
its context: we have mutually established that a die in our 
possession has 3 on every side except for one, which has 
2. The die has been tossed, but we do not know how it 
came up. Then (10) seems like a justified assertion: 


(10) If the die came up even, it came up 2. 


Indeed, the conditional probability of the die coming up 
2, given that it came up even, is 1. In this context, our cre- 
dence that the die came up any even number other than 2 
P(Two) 
P(Two) 
we believe the die to be a standard one, the assertion of 
(10) will be quite unjustified. Believing the die to be stan- 
dard amounts to these credences: P(Even) = 2, P(Two) = %, 
and P(Even a Two) = %. So, P(Even| Two) = 4 %, 


iy 


= 1. But when 


is 0, so P(Even , Two) = P(Two) and 


Since conditional probabilities are only defined when 
the antecedent is assigned a nonzero probability, Adams’ 
thesis is of limited use for counterfactuals. Further, 
indicative and counterfactual pairs often differ in their 
assertability, for example, (3) and (4). To address this, 
Adams (1976) proposed the prior probability analysis of 
counterfactuals: 


Adams’ Prior Probability Analysis The assertability of 
“B would have been, if A had been” is proportional 
to Po(B| A), where Po is the agent’s credence prior to 
learning that A is false. 


Consider a counterfactual variant of (10) uttered in 
the context where the die is known to have 3 on five 
sides and 2 on one, and we have just learned that the 
die came up 3. 


(18) If the die had come up even, it would have come 
up 2. 


Prior to learning that the die came up 3, and so didn’t 
come up even, P(Even| Two) = 1, as just discussed. So the 
prior probability analysis correctly predicts this utterance 
of (18) to be a perfect assertion. As with the indicative, 
this would change if we believed the die to be 12-sided 
instead. 

There are other cases, however, where Adams’ prior 
probability analysis makes incorrect predictions. Edging- 
ton (2004, p. 21) discusses these cases and concludes that 
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the analysis requires an amendment: Po may also reflect 
any facts the agent learns after they learn that the ante- 
cedent is false, provided that those facts are causally inde- 
pendent of the antecedent. Kvart (1986) integrates causal 
information into a different objective conditional prob- 
ability analysis. 

More recently, this focus on causality and probabil- 
ity has inspired a different approach to counterfactu- 
als. Many using probabilistic tools now favor modeling 
causal information in terms of Bayesian networks (Pearl, 
2009). It is then possible to formulate a semantics for 
counterfactuals directly in terms of Bayesian networks. 
As section 4.2 will explain, Bayesian networks have sig- 
nificant advantages to a standard probabilistic repre- 
sentation when trying to formulate a computationally 
tractable representation of an agent’s knowledge about 
the world. 


4.2 Bayesian Networks 

When an agent’s credence in B is the same as their cre- 
dence in B given A and B given —=A,B is probabilistically 
independent of A: 


Probabilistic Independence B is probabilistically inde- 
pendent of A just in case P(B) = P(B| A) = P(B| =A). 


Bayesian networks are built on the mathematical insight 
that it is possible to represent an agent’s credences by 
representing only the conditional probabilities of depen- 
dent variables and the probabilities of independent vari- 
ables. This mathematical insight has been taken to be of 
great importance to artificial intelligence and cognitive 
science. It makes probabilistic representations of agents’ 
beliefs computationally tractable.’* But it also stores 
immensely useful information. It facilitates counterfac- 
tual reasoning, reasoning about actions, and explanatory 
reasoning. 

An agent’s knowledge about a system containing 
eight variables could be represented by the directed acy- 
clic graph and system of structural equations between 
those variables in figure 6.1.1. While the arrows mark 
relations of probabilistic dependence, the equations 
characterize the nature of the dependence; for example, 
“H = Fv G” means that the value of H is determined by 
the value of Fv G (but not vice versa). Consider just the 
three rightmost nodes of figure 6.1.1. They are an appro- 
priate representation for an agent who has credences 
about three propositions, and their probabilistic depen- 
dencies correspond to the indicative conditionals (19a) 
and (19b). Let us further suppose they have the uncon- 
ditional credence corresponding to (19c). 
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(19) a. If both Fran and Greta attend, Harriet attends. 
b. If either Fran or Greta don’t attend, Harriet 
won’t attend. 
c. Fran attended, Greta did not, and so Harriet did 
not. 


d. If Greta had attended, Harriet would have 
attended. 


The counterfactual (19d) seems true in this scenario. 
Pearl’s (2009, chapter 7) analysis can capture this: 


Interventionism Evaluate G>H relative to a Bayesian 
network by removing any incoming arrows to G, set- 
ting its value to 1, and projecting this change forward 
through the remaining network. If His 1 in the result- 
ing network, G > H is true; otherwise, it’s false. 


On this method, one first intervenes on G: remove 
the arrow coming in to G and the equation G = E, and 
replace it with G = 1. One then solves for H using the 
equation H = Fa G. Since intervention does not affect the 
value of F, it remains 1. So, it follows that H = 1 and that 
the counterfactual is true. Pearl (2009, chapter 7) shows 
that the logic of interventionist counterfactuals is very 
close to similarity analyses (D. K. Lewis, 1973; Stalnaker, 
1968)—hence my use of “>” here. 

Unlike similarity analyses, Bayesian networks provide 
explicit models of the knowledge that makes counter- 
factuals true. This allows them to better navigate the 
numerous counterexamples to the similarity analysis 
surveyed in Starr (2019, section 2.5) and provide an 
explicit theory of how counterfactuals are context sen- 
sitive. To be sure, interventionism has limitations and 
faces a number of counterexamples."” But there is now 
a burgeoning interdisciplinary literature refining inter- 
ventionism (Kaufmann, 2013; Lucas & Kemp, 2015; 
Schulz, 2007, 2011) and pursuing alternatives also based 
on Bayesian networks (Hiddleston, 2005; Rips, 2010). 
This is a very active area of research. 


H=FYG 


Figure 6.1.1 


Bayesian network and structural equations. 
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5. Conclusion 


Recent work on the logic of conditionals maintains 
that they have three key properties: they are non-truth- 
functional, they are context sensitive, and their ante- 
cedents are interpreted nonmonotonically. Certain core 
validities, like modus ponens, and invalidities, like mate- 
rial negation and material antecedent, have been captured 
alongside these key properties. Possible-worlds analyses 
have come in two basic varieties: strict analyses and simi- 
larity analyses. These two varieties have been augmented 
in various ways using restrictor analyses of modality and 
dynamic semantics. 

Accounting for the particular contextual features that 
fix the truth-conditions of conditionals remains a chal- 
lenge for these approaches. Probabilistic analyses pre- 
sent a promising option here but are still very much in 
development. 


Notes 


1. While commonly assumed, this division is debated (e.g., 
Dudman, 1988). Others prefer the categories of indicative and 
subjunctive conditionals (e.g., Declerck & Reed, 2001, p. 99; 
von Fintel, 1999). For an overview of these issues, see Starr 
(2019). For a dedicated survey of subjunctive conditionals, see 
von Fintel (2012). For indicatives, see Gillies (2012) and Edg- 
ington (2014). 


2. Here, the “#” is used in the descriptive convention of lin- 
guistics, where it indicates the native speaker judgment that 
the sentences are grammatical but can’t be used in this context. 


3. This chapter will assume that the if-then structure in indica- 
tive and counterfactual conditionals should have the same logical 
analysis and that their difference should be explained in terms of 
their different morphology. Not all philosophers have shared this 
assumption (e.g., D. K. Lewis, 1973), but it remains the default 
one for good reasons. See Starr (2019) for further discussion. 


4. For a more exhaustive and formal survey of conditional 
logics, see Arl6-Costa, Egré, and Rott (2007/2019). 


5. As is often remarked in those textbooks, it is the only truth 
function that captures these features. 


6. While Grice (1989) attempted a pragmatic defense, it has 
never been satisfactorily extended to the problems presented 
here. 


7. For technical reasons not discussed here, it is generally 
assumed in modal logic that there are more possible worlds 
than unique combinations of atomic truth-values. 


8. Kripke (1963), as well as most work in modal logic, actually 
uses accessibility relations R(w,w'). But accessibility functions 
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simplify the presentation and are definable in terms of acces- 
sibility relations: R(w) = { w' | R(w, w’) }. 


9. They also address the “paradoxes of strict implication,” for 
example, 7>O AF A= B. 


10. See D. K. Lewis (1973, section 2.7) for the various formula- 
tions. The set selection formulation makes the limit assump- 
tion: A-worlds do not get indefinitely more and more similar 
to w. D. K. Lewis (1973) rejected this assumption, but it will 
merely simplify exposition here. See Starr (2019) for discussion 
of the limit assumption. 


11. f(w, A) is shorthand for f(w,[A]), where [A] is the set of 
worlds in which A is true. 


12. Pollock (1976) adopts (a) and (b), D. K. Lewis (1973) and 
Nute (1975) adopt (a)-(c), and Stalnaker (1968) adopts (a)—(d). 


13. In fact, weak centering suffices: w €f(w, p) if w Ep. 


14. Where there is no overt modal, Kratzer (1986) proposes 
that conditionals involve a covert epistemic necessity modal. 


15. Moss (2012) and K. S. Lewis (2018) are similarity analyses 
that explain these data pragmatically. 


16. Probabilities are taken to obey the Kolmogorov axioms. See 
Hajek (2019) for details. 


17. As noted in section 1, indicatives are infelicitous when 
their antecedent is known to be false. This is expected on 
Adams’s analysis, since P(B | A) is undefined when P(A) = 0. 


18. A complete description of an agent’s credences involves a 
joint probability distribution over all Boolean combinations of 
the variables. For a system with 8 variables, this requires stor- 
ing 2°=256 probability values, while the Bayesian network 
would require only 18—one conditional probability for each 
Boolean combination of the parent variables and one for each 
of the independent variables. See Sloman (2005, chapter 4) and 
Pearl (2009, chapter 1) for details. 


19. It does not apply to conditionals with logically complex 
antecedents or consequents. This limitation is addressed by 
Briggs (2012), who also axiomatizes the logic and compares 
it to D. K. Lewis (1973) and Stalnaker (1968)—significantly 
extending the analysis and results in Pearl (2009, chapter 7). 
For counterexamples, see Hiddleston (2005) and Rips (2010). 
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6.2 The Suppositional Theory of Conditionals and Rationality 
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Summary 


In a suppositional theory, a conditional, if A then B, is 
interpreted as B, supposing A, and its probability, P(if 
A then B), is the conditional probability of B given A, 
P(B|A). Conditionals are ubiquitous in reasoning and 
essential to it. There is strong experimental support for 
the hypothesis that P(if A then B) = P(B|A) in people’s 
judgments, and this finding opens up the possibility of 
giving a unified Bayesian account of human rational- 
ity, reasoning, and decision making. Probability theory 
can be used to define validity in reasoning, and coher- 
ence intervals can be specified for the conclusions of 
inferences. It is rational, by Bayesian standards, to con- 
form to these intervals, and by this measure, people are 
often rational but sometimes commit fallacies and have 
biases. Investigating the coherence and incoherence in 
people’s reasoning will give us a deeper understanding 
of its rationality. 


1. Conditionals and Supposing 


The suppositional theory of conditionals (Edgington, 
2014) is a contribution to the Bayesian analysis of 
human rationality and its understanding of subjective 
probability judgments as expressing degrees of belief. To 
state the theory informally, it treats a natural-language 
indicative conditional, if A then B, as equivalent to B, 
supposing A, and it identifies the subjective probability 
of the conditional, P(if-A then B), with the probability of 
B given A, so that P(if A then B) = P(B| A). This identity, 
which originates in foundational work on subjective 
probability theory (de Finetti, 1936/1995, 1937/1964; 
Ramsey, 1926/1990, 1929/1990), has such deep implica- 
tions for a Bayesian account of rationality and reasoning 
that it has been called the Equation (Edgington, 1995). 
Given the Equation for the natural-language condi- 
tional, the normative standard for rational reasoning 
with it is Bayesian probability theory (chapter 4.5 by 
Chater & Oaksford, this handbook; Oaksford & Chater, 


2007; Over, 2020; Over & Cruz, 2018). In the psychology 
of reasoning, the Equation becomes the conditional prob- 
ability hypothesis: that people’s probability judgments will 
conform to P(ifA then B) = P(B|A). A suppositional theory 
of the conditional, as we will interpret it, has both norma- 
tive and descriptive aspects. It endorses the Equation as 
setting the correct normative standard for people’s condi- 
tional reasoning and implies the descriptive conditional 
probability hypothesis. Supporters of the theory aim to 
confirm the hypothesis in experimental studies and to 
investigate how far people comply with the standards of 
probability theory in their conditional reasoning. Sup- 
positional theory does not imply this compliance will be 
perfect. Even if people judge that P(if A then B) = P(B| A), 
cognitive limitations will sometimes result in biases and 
fallacious inferences. However, suppositional theory does 
propose that people’s conditional reasoning will be more 
accurately described by probability theory than by binary 
extensional logic (Elqayam & Over, 2013). 

Reasoning from A to B (supplemented with back- 
ground beliefs) can be “summed up” with a conditional 
assertion if A then B, and assertions of if A then B can be 
supported by reasoning from A to B (plus background 
information). Thus, any research program implying that 
P(if A then B) = P(B| A), as a normative principle and as 
broadly descriptively adequate, places conditional prob- 
ability at the center of its account of human reasoning. 
This identity opens up the possibility of integrating the 
study of reasoning with that of judgment and decision 
making, which depends so much on conditional prob- 
ability, so producing an overall account, normative and 
descriptive, of human rationality (for developments 
along these lines, see Oaksford & Chater, 2007, 2020; for 
the debate in psychology about human rationality, see 
chapter 1.2 by Evans, this handbook). 


2. The Conditional Probability Hypothesis 


Psychological research has highly corroborated the con- 
ditional probability hypothesis as descriptive of people’s 
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probability judgments across a wide range of condition- 
als (Evans, Handley, Neilens, & Over, 2007; Evans, Hand- 
ley, & Over, 2003; Fugard, Pfeifer, Mayerhofer, & Kleiter, 
2011; Oberauer & Wilhelm, 2003; chapter 4.6 by Ober- 
auer & Pessach, this handbook; Over, Hadjichristidis, 
Evans, Handley, & Sloman, 2007; Pfeifer & Kleiter, 2005; 
Singmann, Klauer, & Over, 2014). Its intuitive appeal 
can be appreciated by considering these examples: 


(1) If the fair coin is tossed (F), then the result will be 
Heads (A). 


(2) If the double-headed coin is tossed (D), then the 
result will be Tails (T). 


The above are singular conditionals. They are about spe- 
cific events: the next toss of the fair, or the double- 
headed, coin. A general conditional is about a whole 
domain of objects, as in “If an animal is a bird, then it 
can fly.” There are clearly counterexamples to this con- 
ditional in its domain, there being birds that cannot fly. 
A singular conditional with a similar topic would be, “If 
the next animal we see is a bird, then it will be able to 
fly.” This conditional could be highly probable, depend- 
ing on where we are in the world, and could turn out to 
be true. We will focus on singular conditionals like (1) 
and (2) in this chapter (see Cruz & Oberauer, 2014, on 
general conditionals). 

Intuitively, the probability of (1), P(if F then H), is .5, 
and the probability of (2), P(if D then T), is 0. These are 
the correct judgments by the Equation, and those that we 
would predict using the conditional probability hypoth- 
esis, since P(H|F) = .5 and P(T|D) = 0. These judgments 
are not, however, correct and predictable if the natural- 
language conditional is interpreted as the material con- 
ditional of binary extensional logic, P(if A then B) = 
P(ADB) = P(not-A or B). The material conditional ADB 
is logically equivalent to the disjunction not-A or B, 
and similarly, not-ADB is logically equivalent (not-not-A 
becoming A) to A or B. However, the identification of P(if 
A then B) with P(not-A or B) is most counterintuitive, and 
the same can be said for identifying P(ifnot-A then B) with 
P(A or B). Assume the coins referred to in (1) and (2) are 
valuable ancient Roman examples in a locked museum 
display case. Then the probabilities that they will be 
tossed, P(F) and P(D), are very low, making P(not-F) and 
P(not-D), and so P(not-F or H) and P(not-D or T), very high. 
But it is absurd to claim that the probabilities of (1) and 
(2) are very high merely because these coins are highly 
unlikely to be tossed, and it is difficult to see how ratio- 
nal reasoning and rational decision making, with natural- 
language conditionals if A then B, can be given a unified 
treatment when P(if A then B) = P(ADB) = P(not-A or B). 
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The logical problem with identifying, for instance, 
P(if F then H) and P(not-F or H) is that it is valid to infer 
not-F or H from not-F, which entails that P(not-F) < P(not- 
F or H), forcing P(not-F or H) to be high whenever P(not- 
F) is high. A truth table can be used in classical logic to 
prove that it is logically valid to infer A or B from A, an 
inference termed or-introduction. Table 6.2.1 is the truth 
table for A or B, and it clearly shows that A or B is true 
when A is true, making the inference from A to A or B 
valid, with the further normative result that P(A) < P(A 
or B). Table 6.2.2 is the truth table for the material con- 
ditional ADB or, equivalently, not-A or B, which validly 
follows from not-A. But to claim that it is valid to infer 
a natural-language conditional if A then B from not-A is 
one of the “paradoxes” of the material conditional (Edg- 
ington, 2014; Evans & Over, 2004). Its validity entails 
that P(not-A) < P(ifA then B), which produces the highly 
counterintuitive results we have seen in the example 
above, and people have been found to reject the validity 
of this inference in their probability judgments (Cruz, 
Over, & Oaksford, 2017; Pfeifer & Kleiter, 2011; Politzer 
& Baratgin, 2016). 

The only major theory in the psychology of reason- 
ing that closely related (until recently) people’s repre- 
sentation of the natural-language conditional to the 
material conditional was mental model theory, with the 
result that the paradoxes of the material conditional 
were “valid” in this theory (Johnson-Laird & Byrne, 


Table 6.2.1 
The truth table for the disjunction A or B 


A B Aor B 
1 1 1 
1 0 1 
0 a I 
0 0 0 


Note: 1 = true; O = false. 


Table 6.2.2 
The truth table for the material conditional A 5 B, equivalent 
to not-A or B 


A B ADB 
1 1 1 
1 0) 0 
0 1 1 
0 0 1 


Note: 1 = true; 0 = false. 
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1991, pp. 73-74). But counterintuitive implications like 
those described above have led to a radical revision of 
mental model theory, in which the paradoxes are no 
longer considered valid (Johnson-Laird, Khemlani, & 
Goodwin, 2015). The revision is in a state of develop- 
ment (chapter 2.3 by Johnson-Laird, this handbook), 
and we will not consider it here (but see Baratgin et al., 
2015; Oaksford, Over, & Cruz, 2019; Over, 2020; Over & 
Cruz, 2018). 

Support for the conditional probability hypothesis 
also disconfirms the claim (Johnson-Laird & Byrne, 
1991, p. 74) that a natural-language conditional can be 
given the same truth table as the material conditional 
(table 6.2.2). This table entails that P(ADB) = P(not-A 
or B), but P(B| A) = P(not-A or B) only in certain ex- 
treme cases (Gilio & Over, 2012), for example, when 
P(A) = P(B) = 1. Four possibilities are displayed in 
table 6.2.2: A&B, A &not-B, not-A &B, and not-A &not-B. 
As an example, assume we judge that these possi- 
ble states have the same probability: P(A&B) = P(A& 
not-B) = P(not-A & B) = P(not-A & not-B) = .25. We now 
clearly see that P(B|A) = P(A &B)/P(A) =.5, but P(not-A 
or B) = P(A &B) + P(not-A & B) + P(not-A &not-B) = .75. More 
informally, the problem is that not-A or B makes a dis- 
junctive statement about the not-A-possibilities, with 
not-A or B true in those possibilities, and so the prob- 
ability of these not-A-cases contributes to P(not-A or B) 
but not to P(B|A). Lewis (1976) proved a much more 
general result. He considered conditionals like those in 
Lewis (1973) and Stalnaker (1968). We might call these 
Lewis-style conditionals and symbolize them with if A 
then, B, which means that B holds in the closest possi- 
ble world (or worlds) in which A holds. In this way, if 
A then, B can hold in a not-A-world, and that implies 
that P(if A then, B) can be affected by P(not-A). Because 
of that, Lewis was able to prove that P(if A then, B) can- 
not generally be identified with P(B| A), which depends 
only on A-worlds. 


3. The Ramsey Test and the de Finetti 
and Jeffrey Tables 


A suppositional conditional is not a material conditional 
or a Lewis-style conditional (Edgington, 2014). In sup- 
positional theory, ifA then B means B, supposing A, which 
does not make any statement about, or depend upon, 
not-A-possibilities. We can make the identification that 
P(if A then B) = P(B| A) in a suppositional theory, and so 
this conditional does satisfy the Equation. It can simply 
be called a suppositional conditional, but it has also been 
termed a probability conditional (Adams, 1998) and a 
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conditional event (de Finetti, 1936/1995, 1937/1964). 
We can assess P(if A then B) for a suppositional condi- 
tional by using the Ramsey test, which was first described 
by Ramsey (1929/1990) and later extended by Stalnaker 
(1968). In this psychological process, people make a judg- 
ment about P(if A then B) by supposing A, while making 
minimal changes to preserve (or restore) consistency in 
their beliefs, and then judging their degree of belief in B 
under this supposition, yielding their subjective condi- 
tional probability for B given A, P(B| A). The Ramsey test 
has had a major impact in philosophy, psychology, and 
other parts of cognitive science (Edgington, 1995; Evans & 
Over, 2004; Oaksford & Chater, 2007; Pearl, 2013). 

We can explain the Ramsey test more fully with an 
example: assume we are in a car and trying to make a 
rational decision about which route to take home one 
particular afternoon. We are discussing the following 
two conditionals relevant to this decision: 


(3) If we take the short city route (S), it will be a quick 
journey (Q). 

(4) If we take the long countryside route (L), it will be an 
enjoyable journey (E£). 


The short city route home is an uninteresting drive, 
but the long countryside route can be a pleasant drive. 
Of course, it is not certain that the city route will be 
quick and the countryside route enjoyable. A number of 
uncertain factors can affect these outcomes: traffic jams, 
changing weather conditions, and accidents. For a ratio- 
nal decision, we must assess both P(if S then Q) and P(if 
L then E), and we can do that via the Ramsey test and 
the Equation. We might make the first Ramsey test on 
(3), by supposing S and making a judgment about how 
likely Q is to follow, and the second on (4), by suppos- 
ing L and making a judgment about how likely E is to 
follow. Perhaps in both tests, we judge that traffic jams 
and so on are unlikely on this particular day, and so we 
end the tests by judging that P(Q| S$) and P(E|L) are both 
high, giving us high confidence in (3) and (4). A rational 
decision would then depend on our relative desires, that 
particular afternoon, for a quick drive as opposed to a 
pleasant one. 

Let us say that, after some pondering, we decide that 
afternoon to take the longer but more pleasant drive, 
and we set off on the long countryside route. Now our 
probability judgments change, and we become certain 
that S is false: P(not-S) becomes 1. In this case, Ramsey 
(1929/1990) said that we would lose interest in (3), 
which he described as becoming “void,” and de Finetti 
(1936/1995, 1937/1964) also took this view and used 
“void” in this way. We would not use (3) when we were 
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on the countryside route and still less an indicative condi- 
tional beginning, “If we are on the city route.” We could, 
though, take an interest in and assert a counterfactual: 


(5) If we had taken the short city route home, it would 
have been a quick journey. 


Counterfactuals are a topic of considerable interest 
in cognitive, social, and developmental psychology. For 
instance, Kahneman and Miller (1986) established the 
interesting result that, when the city route is our usual 
way home, we would feel more regret when we had an 
accident on the countryside route than when we had 
an accident on the city route. Our relatively intense 
regret after an accident on the countryside route and 
our high confidence in counterfactual (5), which would 
come from a Ramsey test on (5) similar to the one on (3), 
would be closely linked. This regret could affect, ratio- 
nally or irrationally, our future decision making about 
which route to take. With space limitations, we will, 
however, continue to focus on indicative condition- 
als, like (1)-(4), in the rest of this chapter (but see Over, 
2017; Over & Cruz, 2019). 

In de Finetti’s account of indicative conditionals, if 
A then B is true when A and B hold, false when A holds 
and B does not, and “void” when A does not hold. These 
three values can be represented in what has been called 
a de Finetti table, and we display this as table 6.2.3. In 
traditional research on truth tables in the psychology 
of reasoning, participants produce a so-called defective 
table that arguably matches table 6.2.3 and confirms 
de Finetti’s analysis (Baratgin, Over, & Politzer, 2013; 
Over & Baratgin, 2017; Politzer, Over, & Baratgin, 2010). 
As we have already noted, Ramsey also used “void” for an 
indicative conditional with an antecedent known to be 
false. He added that we lose interest in these indicative 
conditionals “except as a question about what follows 
from certain laws or hypotheses” (Ramsey, 1929/1990, 
p. 155). This suggests that conditionals that follow 
from logical laws, such as if A&B then B, are not void 
but rather certainly true (Baratgin et al., 2013), and the 
Ramsey test itself implies that, in general, a conditional 
with a false antecedent is only void in not having a fac- 
tual truth value, and that it has its conditional probabil- 
ity as a subjective value. A table in which if A then B has 
the value P(B| A) when A is false has been called a Jeffrey 
table (after Jeffrey, 1991), and it is found, explicitly or 
implicitly, in contemporary followers of de Finetti and 
Ramsey (Cruz & Oberauer, 2014; Over & Baratgin, 2017; 
Over & Cruz, 2018; Pfeifer & Kleiter, 2009). Table 6.2.4 is 
the Jeffrey table for if A then B. 
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Table 6.2.3 
The de Finetti table for if A then B 


A B if A then B 
1 1 1 

1 0 0 

0) 1 Vv 

0 0 Vv 

Note: 1 = true; 0 = false; V = void. 

Table 6.2.4 

The Jeffrey table for if A then B 

A B if A then B 
1 1 a 

1 0 0 

0 1 P(B\A) 

0 ) P(BIA) 


Note: 1 = true, 0 = false; P(B| A) = the subjective conditional 
probability of B given A. 


4. Rational Reasoning from Degrees of Belief 


The suppositional theory of conditionals employs the 
Ramsey test and de Finetti and Jeffrey tables to give a 
probabilistic account of conditionals. It rests on the 
foundational work of Ramsey and de Finetti on condi- 
tionals and, more generally, on their identification of 
people’s probability judgments with their degrees of 
belief. With its focus on degrees of belief, suppositional 
theory is a prime example of the new Bayesian paradigm 
in the psychology of reasoning (Elqayam & Over, 2013). 
Contributions to the traditional paradigm were primar- 
ily concerned with inferences from premises that were 
supposed to be assumed true, which meant that these 
were, in effect, taken to be certain, and the relevant nor- 
mative standard was binary extensional logic (Johnson- 
Laird & Byrne, 1991; Over & Cruz, 2018). However, most 
human inferences, in ordinary affairs and science, are 
from uncertain beliefs or from hypotheses that, though 
plausible, are not certain. This uncertainty has to be 
taken into account in reasoning to arrive at a rational 
degree of confidence in a conclusion. 

For instance, after we decide to take the countryside 
route home and set off on it, we can use the valid infer- 
ence form of modus ponens (MP) to infer that we will have 
an enjoyable journey from (4) as an additional premise. 
More formally, we would use MP to infer E from if L then 
E as the major premise and L as the minor premise. We 
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would not, though, treat (4) as certain or simply assume 
it. Possible disabling conditions, like a sudden change in 
the weather, could make (4) false. By the Ramsey test, (4) 
has a probability that is equal to P(E| L), which is high but 
is not 1, with 1 representing certainty. 

But let us consider first the simpler case of a one- 
premise inference, rather than a two-premise inference, 
like MP. In their classic article on conjunction, Tversky 
and Kahneman (1983) pointed out that, because it is 
logically valid to infer B from A & B, people should judge 
that P(B) = P(A &B). Tversky and Kahneman relied on 
a necessary relation between logical validity and coher- 
ent probability judgments, which conform to the rules 
of probability theory. To be incoherent by violating the 
principle that P(B) = P(A &B) in one’s probability judg- 
ments is to commit what Tversky and Kahneman called 
the conjunction fallacy. 

The coherence of probability judgments can be used 
to define validity. We can say that a one-premise infer- 
ence is probabilistically valid, p-valid, if and only if the 
probability of the conclusion can never be coherently 
lower than the probability of the premise. For an infer- 
ence with two or more premises, we first define the 
uncertainty of a statement as 1 minus its probability. For 
example, when P(A) = .6, the uncertainty of A is .4. Then 
an inference is p-valid if and only if, for all coherent 
probability assignments, the uncertainty of the conclu- 
sion is never greater than the sum of the uncertainties 
of the premises (Adams, 1998). A p-valid inference never 
increases the uncertainty present in its premises, and it 
differs from a strong inductive argument in precisely 
this respect (Evans & Over, 2013). 

The one-premise inference from A &B to B is known 
as &-elimination in logic, and we can ask what rational 
degree of confidence we should have in B when there 
is some uncertainty about A&B. The answer given 
above is that our degree of confidence should be such 
that P(B) = P(A & B). More formally, for P(A & B) = x, the 
coherence interval for this inference is [x, 1], meaning that 
P(B) should lie between x and 1 inclusive. 

Coherence intervals can also be derived for inferences 
with more than one premise. The coherence interval for 
MP follows from the Equation and the total probability 
theorem of probability theory. Consider (4) again, infer- 
ring E from if L then E and L. By the total probability 
theorem, 


(6) P(E) = P(L)P(E|L) + P(not-L)PCE | not-L). 


When we definitely decide to take the long countryside 
route home, we judge P(L) = 1, and we can infer from 
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(6) that our confidence that we will have an enjoyable 
journey is P(E|L), which we might assess as .8 since 
there is some modest probability of a disabling condi- 
tion that could make our journey unpleasant. Other 
people might agree that P(E|L) =.8, but not knowing 
what we have decided, perhaps set P(L) = .5. Using (6), 
they can then compute P(L)P(E | L) = (.5)(.8) = .4. When 
P(L) = .5, P(not-L) is of course also .5. The last part of 
equation (6), P(E|not-L), might be unknown. But by 
probability theory, it has to lie between 0 and 1. When 
it is 0, P(E) = P(L)P(E| L) = .4. And when it is 1, P(E) = 
P(L)P(E | L) + P(not-L) = .9. To be coherent, the probabil- 
ity of the conclusion of MP for these people must then 
lie in the interval [.4, .9], that is, between .4 and .9. 

The derivations of coherence intervals for inferences 
(Evans, Thompson, & Over, 2015; chapter 4.4 by Pfeifer, 
this handbook; Pfeifer & Kleiter, 2009) yield a norma- 
tive theory for the rational degrees of confidence people 
should have in the conclusions they infer from uncer- 
tain premises. To fall outside the coherence interval for 
an inference is to be incoherent and exposed to a Dutch 
book (i.e., a bet that will be lost no matter which possible 
outcome occurs). Dutch book arguments are the founda- 
tion of the normative Bayesian theory of coherence and 
rationality (Howson & Urbach, 2005). 

People are not always coherent in their degrees of 
belief, but it is quite plausible that explicit inference 
would help them to be more rational at this basic level, 
for otherwise what benefit can come from the time 
and effort that go into explicit inference? Studies have 
shown that people are sometimes more coherent in the 
degrees of belief that they have in the conclusions of 
explicit inferences, but not necessarily (Cruz, Baratgin, 
Oaksford, & Over, 2015; Cruz, Over, Oaksford, & Barat- 
gin, 2016; Cruz et al., 2017; Evans et al., 2015; Singmann 
et al., 2014). A striking example of when explicit infer- 
ence does not facilitate coherence is a context in which 
the conjunction fallacy occurs (Tversky & Kahneman, 
1983). In this context, people can incoherently judge 
P(B) < P(A&B) even when they explicitly infer B from 
A&B (Cruz et al., 2015) and even though their judg- 
ments for the same inference are coherent when using 
neutral contents (Cruz et al., 2015; Politzer & Baratgin, 
2016). This finding underlines the robustness of the fal- 
lacy and the importance of accounting for it (Costello & 
Watts, 2014; Tentori, Crupi, & Russo, 2013). 

A further important inference form for the psychol- 
ogy of reasoning is centering. One-premise centering is 
sometimes called conjunctive sufficiency: it is the infer- 
ence from A&B as a single premise to if A then B asa 
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conclusion. Two-premise centering is the inference from 
A and B as separate premises to if A then B as a conclu- 
sion. Centering is a valid inference for a wide range of 
conditionals, including the material conditional, Lewis- 
style conditionals, and the suppositional conditional, 
and there is evidence that people conform to it in their 
degrees of belief (Cruz et al., 2015; Cruz et al., 2016; 
Pfeifer & Tulkki, 2017; Politzer & Baratgin, 2016). In con- 
trast, the inference is invalid in inferentialist theories of 
the conditional (Douven, 2015, 2016; Douven, Elqayam, 
Singmann, & van Wijnbergen-Huitink, 2018). In seman- 
tic versions of these theories, if A then B means that there 
is a deductive, inductive, or abductive link between A 
and B, and if'A then B can only be true when there is such 
a link between A and B. Supporters of this position argue 
that centering will be rejected for missing-link condition- 
als. These are conditionals if A then B with no relation 
between A and B, and the claim is that these condition- 
als do not follow from the truth of A & B. The following 
is an example of a missing-link conditional: 


(7) If it is sunny in Canberra (C) today, then it is raining 
in London (N). 


Supposing that C and N are both true, and so C&N is 
true, inferentialists would argue that (7) does not follow. 
It certainly could be pragmatically very odd in a discus- 
sion with someone to begin with C and N as premises, 
for the purpose of explicitly inferring (7) as a conclusion. 
On the other hand, if participants in an experiment are 
given a missing-link conditional ifA then B first, followed 
by confirmation of A and B, they hold that if A then B 
is true (Skovgaard-Olsen, Kellen, Krahl, & Klauer, 2017, 
figure 1). Indeed, if we were to assert (7) first in a discus- 
sion, however irrelevantly or oddly, and were to place a 
bet on it, we would surely claim to have stated a truth 
and to have won our bet when C and L turned out later 
to be true. This claim is in accordance with the de Finetti 
and Jeffrey tables for if A then B and a bet on it, and it has 
been strongly supported empirically in truth table tasks 
(see also Evans et al., 2003; Oberauer & Wilhelm, 2003), 
but it goes against the assumptions of inferentialism. 

On the other hand, it is hard to see how (7), or any other 
missing-link conditional, can be used in rational decision 
making. When there is no epistemic relation between 
A and B, making A and B independent, P(B|A) = P(B), and 
A cannot be a reason for doing B. There is evidence that 
in some contexts, people conform to the Equation more 
often when P(B|A) > P(B|not-A), suggesting an inter- 
pretation of if A then B in which A raises the probability 
of B (Skovgaard-Olsen, Singmann, & Klauer, 2016), but 
there is also evidence against this prediction (Oberauer, 
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Weidenfeld, & Fischer, 2007; Over et al., 2007; Singmann 
et al., 2014). Cruz et al. (2016) find that the lack of an 
epistemic relation between A and B affects probability 
judgments not only about conditionals but also about 
conjunctions A &B and disjunctions A or B. This implies 
that a missing-link effect on probability judgments is not 
specific to conditionals, suggesting that it is pragmatic 
and not semantic. An effect that is not specific to con- 
ditionals cannot be used to distinguish the meaning of 
conditionals from that of other statement types. The 
findings of Cruz et al. (2016) also suggest that having 
a common topic of discourse between A and B is more 
important for the pragmatic acceptability of statements 
than a specific epistemic relation (see, on this, Over & 
Cruz, 2018; for a related account of why the computation 
of subjective probabilities for compound events is more 
difficult when A and B come from different domains, see 
Sanborn & Chater, 2016). The uselessness of missing-link 
conditionals for rational decision making does not have 
to be a matter of semantic meaning. 


5. Rational Belief Updating 


Suppositional reasoning is fundamental to rational deci- 
sion making and to the belief revision and updating that 
goes with it. In scientific Bayesian inference, we might 
assign a low prior probability to a hypothesis H at time 
1, making P,(H) low, and infer (via Bayes’ theorem) a 
high probability of H supposing we get an finding F in 
an experiment, making P,(H|F) high. Assume we now 
run an experiment at time 2 and learn that F holds (and 
nothing else relevant). We can then use the new infor- 
mation about F to update our degree of belief in H at 
time 2, P2(H) = P:\(H|F), in an inference termed (strict) 
conditionalization (Howson & Urbach, 2005). 

In a suppositional theory, we can identify this Bayes- 
ian process with conditional reasoning over time, in 
which we begin with P,(H) low and P,(if F then H) high, 
and then after we run the experiment and find that P2(F) 
holds, we can infer that P2(H) is high using dynamic MP 
(Oaksford & Chater, 2013): inferring high confidence in 
H from high confidence in if F then H after learning F as a 
result of acquiring new information. Bayesian inference 
like this is often an essential part of ordinary rational 
decision making. 

A little more formally, suppose we begin at time 1 
with a degree of confidence in H, say P;(H) =.5, and 
in H given F, say P\(H|F) = P(if F then H) = .8. At a later 
time 2, after acquiring new information, we judge that 
P,(F) = 1, and we can then infer that P2(H) = .8. When 
we are less sure of F at time 2, say P2(F) = .7, we can use 
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Jeffrey conditionalization (Jeffrey, 1983) to infer what our 
confidence in H should be at time 2: 


(8) P2(A) = P(F)P\(A | F) + P2(not-F)P,(H | not-F). 


Jeffrey conditionalization is derived from the total 
probability theorem, an instance of which is (6). Both it 
and (strict) conditionalization depend on the assumption 
that the conditional probabilities are invariant (Oaks- 
ford & Chater, 2013), which means that P,(H|F) = 
P,(H|F) and P,(H|not-F) = P,(H|not-F). People do not 
always conform to Jeffrey conditionalization (Hadjichris- 
tidis, Sloman, & Over, 2014; Zhao & Osherson, 2010), 
perhaps because they tend to focus on the component— 
either P2(F)P\(H|F) or P2(not-F)P\(H | not-F)—that has the 
higher probability and do not take into account the less 
probable one. 

The problem of making a precise decision about 
P,(H | not-F) can be bypassed by considering the coher- 
ence interval: the minimum value of P;(H|not-F) is 0, 
and the maximum value is 1, allowing us to derive, as 
above, a coherence interval for Jeffrey conditionaliza- 
tion, justified when invariance holds. How far people’s 
dynamic reasoning and belief updating are coherent, 
and their subjective conditional probabilities invariant, 
are important questions for future research. 


6. Conclusion 


In suppositional theory, a natural-language indicative con- 
ditional, if'A then B, is equivalent to B, supposing A, and its 
probability is the subjective conditional probability of its 
consequent given its antecedent: P(if A then B) = P(B|A). 
This theory is a central part of a fully Bayesian approach 
to the psychology of reasoning and of judgment and 
decision making, which aims to unify these two subject 
areas and to study how degrees of belief are revised and 
updated in human reasoning, as well as how this process 
affects human decision making. The underlying standard 
of rationality in this approach, for both reasoning and 
decision making, is coherence, which means consistency 
with probability theory. People are sometimes coherent 
and sometimes incoherent in their reasoning and deci- 
sion making, and unifying the two subject areas should 
lead to deeper understanding of why this is so. 
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6.3 Conditional and Counterfactual Reasoning 


Ruth M. J. Byrne and Orlando Espino 


Summary 


We consider a set of questions about conditionals and 
counterfactuals that have led to experimental discover- 
ies able to distinguish between competing explanations 
of human reasoning. The first part of the chapter focuses 
on reasoning with factual conditionals, such as “If she 
made a good painting, she got a treat,” and addresses 
two related questions: (1) Do people think about what 
is possible or what is probable when they understand a 
conditional? and (2) When do people infer a conditional 
relation? The second part of the chapter focuses on rea- 
soning with counterfactual conditionals, such as “If she 
had done everything right, she would have passed her 
driving test,” and also addresses two questions: (1) Do 
people make similar inferences from factual and coun- 
terfactual conditionals? and (2) Do people construct 
“embodied” mental representations of counterfactuals? 
The discoveries made from experimental investigations 
of these questions provide insights into how the mind 
accomplishes reasoning. 


1. Conditionals and Counterfactuals 


Conditional inferences are at the very center of everyday 
human reasoning. Even the youngest child who believes 
“Tf I make a good painting, I will get a treat,” on hear- 
ing that her painting is good, will expect a treat. People 
make conditional inferences often in everyday life. 
Some seem immediate and obvious. Other conditional 
inferences require people to mull over them, to consider 
different possibilities. An office worker who knows “If 
I leave the office at 5 p.m., I will get home in time for 
dinner,” on finding himself still at his desk as the clock 
strikes 5 p.m., may conclude that he will not get home 
on time—or, alternatively, he may begin to think about 
other routes to ensure he makes it. And people make 
inferences even more readily from counterfactual condi- 
tionals, about what once was possible but is so no longer. 
A learner driver who believes “If I had done everything 


right, I would have passed my driving test,” reflecting on 
the fact that she did not pass, will conclude readily that 
she must have done something wrong. Inferences from 
counterfactuals occupy a special place at the center of 
conditional reasoning. 

Conditionals and counterfactuals provide a unique 
window onto the human mind. They have been the 
subject of many intriguing and often conflicting analy- 
ses, especially in philosophy (e.g., Adams, 1975; Lewis, 
1973; Stalnaker, 1968) and psychology (e.g., Evans & 
Over, 2004; Johnson-Laird & Byrne, 2002; Oaksford & 
Chater, 2007; Sloman & Lagnado, 2005). There have 
been many striking experimental discoveries about con- 
ditional inference (for a review, see Nickerson, 2015) and 
about counterfactual inference (for a review, see Byrne, 
2016). Our goal in this chapter is to consider just a few 
of the crucial questions about them that have led to 
important findings since they first became the subject 
of intensive psychological study in the 1960s and 1970s. 
The discoveries provide important insights not only into 
how the mind accomplishes reasoning but also how the 
mind supports other sorts of thinking, including think- 
ing that may seem quite remote from reasoning, such as 
creative imagination. People often create “if only...” 
thoughts about how the past could have turned out 
differently and “what if...” thoughts about how the 
future could be different (e.g., Roese & Epstude, 2017; 
Walsh & Byrne, 2004). Remarkably, the creation of such 
imagined alternatives to reality appears to depend on 
the same cognitive processes that support conditional 
and counterfactual reasoning (Byrne, 2005). 


2. Conditional Reasoning 


The frequency with which people make some condi- 
tional inferences is very high. Almost all participants 
in experiments, when they are told “If he wrote a good 
essay, he got a high mark” and “He wrote a good essay,” 
make the modus ponens inference “He got a good 
mark” (see Nickerson, 2015). The frequency of other 
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conditional inferences varies. Usually about half to two 
thirds of participants, when they are told “He got a high 
mark,” make the affirmation of the consequent infer- 
ence “He wrote a good essay” (see Schroyens, Schaeken, 
& d’Ydewalle, 2001). Often about half of participants, 
when they hear “He did not get a high mark,” make the 
modus tollens inference “He did not write a good essay” 
whereas the other half say that nothing follows. Simi- 
larly, when they hear “He did not write a good essay” 
about half to two thirds of participants make the denial 
of the antecedent inference, “He did not get a good 
mark” (see table 6.3.1). 

How the human mind carries out reasoning is con- 
tended, and there is as yet no agreement on the cog- 
nitive processes that underlie conditional inference. 
An early set of theories proposed that people access a 
mental logic that consists of abstract inference rules, 
such as “If p then q; p; therefore q,” and they construct a 
mental derivation of a conclusion, akin to a logical proof 
(e.g., Braine & O’Brien, 1998; Rips, 1994). On this sort 
of theory, people make modus ponens readily because it 
corresponds directly to an inference rule in their mental 
logic; they have difficulty with modus tollens because 
there is no rule in their mental logic corresponding to 
it, and they must instead construct a mental derivation 
using related rules. Early psychological studies of con- 
ditional reasoning, influenced by long-standing philo- 
sophical analyses of the validity of inferences, assumed 
propositional and predicate logics as the normative stan- 
dard against which to compare human reasoning (e.g., 
Jeffrey, 1981). Another early set of theories proposed 
that the mind contains reasoning modules that consist 
of domain-sensitive inference rules, specialized for con- 
tent such as obligation and permission, or social regula- 
tions of costs and benefits (e.g., Cheng & Holyoak, 1985; 
Cosmides, 1989). Domain-sensitive rule theories limited 
their explanations of conditional reasoning to perfor- 
mance in the Wason selection task (see Ragni, Kola, & 
Johnson-Laird, 2018). They proposed that people make 
inferences akin to modus ponens readily because the 
schemas for most domains contain an inference rule 
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corresponding to it, for example, “If the action is to be 
taken, the precondition must be met”; they make infer- 
ences akin to modus tollens less often because few sche- 
mas contain an inference rule for it. These two sorts of 
theories—abstract and domain-sensitive inference rule 
theories—have few active champions in contemporary 
research on reasoning. 

Nowadays, two other theories are hotly debated, one 
based on Bayesian probability and the other on mental 
models. The probabilist approach proposes that people 
make conditional inferences by relying on Bayesian 
probability calculations—different versions have been 
based on suppositions (e.g., Evans & Over, 2004), proba- 
bilistic logic (e.g., Oaksford & Chater, 2007), and causal 
Bayesian networks (e.g., Lucas & Kemp, 2015; Sloman & 
Lagnado, 2005). They propose that people make infer- 
ences from a conditional such as “If there are apples, 
there are oranges” by assessing conditional probability 
(the probability of an event occurring given that another 
event has already occurred). People compare the prob- 
ability that there are oranges given there are apples, 
P(B|A), to the probability that there are no oranges 
given there are apples, P(not-B|A), and they do not 
think about the situations in which there are no apples 
(Evans, Handley, & Over, 2003). They make the modus 
ponens inference if their prior beliefs indicate P(B | A) is 
greater than P(not-B| A); they have difficulty with the 
modus tollens inference because they have not thought 
about the probability of situations in which there are no 
apples. On this view, probability logic is viewed as the 
normative standard against which to compare human 
reasoning (e.g., Cruz, Baratgin, Oaksford, & Over, 2015). 

In contrast, the mental model theory proposes that 
the cognitive processes that underlie reasoning con- 
struct iconic simulations that correspond to the way 
the world would be if the assertion was true (e.g., Byrne 
& Johnson-Laird, 2009; Johnson-Laird & Byrne, 1991). 
People understand the conditional by envisaging a sin- 
gle possibility at the outset, “There are apples and there 
are oranges” (Johnson-Laird & Byrne, 2002). If need be, 
they can “flesh out” their models to think about other 


Four inferences from the conditional “If he wrote a good essay, he got a high mark” (If A then B) 


Inference Minor premise Conclusion Form 
Modus ponens He wrote a good essay He got a high mark A.B 
Affirm consequent He got a high mark He wrote a good essay B.A 
Modus tollens He did not get a high mark He did not write a good essay Not-B .. not-A 
Deny antecedent He did not write a good essay He did not get a high mark Not-A .*. not-B 
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possibilities that are consistent with the conditional, 
such as “There are no apples and no oranges.” The pos- 
sibilities are conjunctive, that is, it is possible that there 
are apples and there are oranges, and it is possible that 
there are no apples and there are no oranges (e.g., Knhem- 
lani, Byrne, & Johnson-Laird, 2018). People interpret 
conditionals in many different ways—and so condition- 
als cannot be reduced to the truth-functional material 
implication of propositional logic (e.g., Johnson-Laird 
& Byrne, 2002). Instead, the normative standard against 
which to compare human reasoning is the semantic 
principle that an inference is valid if there are no coun- 
terexamples, that is, it is impossible for the premises to 
be true and the conclusion false (e.g., Johnson-Laird, 
Khemlani, & Goodwin, 2015). People make the modus 
ponens inference readily because it corresponds to the 
single possibility they have thought about at the outset; 
they find modus tollens more difficult because it requires 
them to flesh out their models to be more explicit. 

Each of the theories predicts the difference between 
modus ponens and modus tollens, which they were 
designed to explain, but they make different predic- 
tions for other inferences (see Byrne & Johnson-Laird, 
2009, for some examples). Difficulties in comparing 
the two sorts of theories arise because some probabilist 
accounts lack an algorithmic-level specification of the 
mental representations and cognitive processes that the 
human mind relies on to reason (see Oaksford, Over, 
& Cruz, 2019). Nonetheless, two crucial questions have 
led to discoveries that help to distinguish between the 
theories. 


2.1. Do People Think about What Is Possible 

or What Is Probable? 

According to the mental model theory, when people 
understand “if,” they think about what is possible 
(Johnson-Laird & Byrne, 2002). They understand the 
conditional “If there are apples, there are oranges” by 
envisaging initially just a single possibility, in line with 
a principle of parsimony, because of working-memory 
constraints. Their models are small-scale dynamic simu- 
lations of the imagined possibility, but for convenience, 
they can be represented here in the following simple 


diagram: 
apples oranges 


The three dots in the diagram indicate that there may 
be other possibilities, and people can “flesh out” their 
models to think about some of them, for example, 
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apples 
no apples 


oranges 
no oranges 


Reasoners come to many different interpretations of “if,” 
and some of them flesh out their initial representation 
to correspond to this biconditional interpretation, as if 
the conditional asserted “If and only if there are apples, 
then there are oranges.” Others flesh out their models to 
include a further possibility: 


apples oranges 
no apples no oranges 
no apples oranges 


which corresponds to a conditional interpretation of “if” 
(see table 6.3.2). In line with a principle of truth, people 
do not tend to think about the possibilities that are ruled 
out by an assertion. The conditional rules out as impos- 
sible one situation, the one in which there are apples 
and there are no oranges. The biconditional rules out 
as impossible two situations, the one in which there are 
apples and there are no oranges and another in which 
there are no apples and there are oranges. People do not 
tend to think about these impossibilities. 

On the probabilist account, people understand “if” 
by assessing conditional probability (e.g., Evans & Over, 
2004; Oaksford & Chater, 2007). To do so, they carry 
out a Ramsey test (after Ramsey, 1929/1990), in which 
they hypothetically suppose the “if” clause to be the 
case, make changes to their beliefs as needed, and assess, 
on this basis, the probability of the “then” clause. They 
compare the strength of their prior belief that there 
are apples and there are oranges to the strength of 
their prior belief that there are apples and there are no 
oranges (Evans et al., 2003). In other words, they think 
about the probability of a situation that is true accord- 
ing to the conditional and compare it to a situation the 
conditional rules out as false. To work out the likelihood 


Table 6.3.2 

The situations people think about for a conditional “If there 
are apples, then there are oranges” and a biconditional “If and 
only if there are apples, then there are oranges” according to 
two theories 


Probability and prior 


Mental models beliefs 


Conditional apples and oranges 
no apples and no oranges 


no apples, and oranges 


apples and oranges 
apples and no oranges 


Biconditional apples and oranges 


no apples and no oranges 


apples and oranges 
apples and no oranges 
no apples, and oranges 
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that if there are apples, then there are oranges, people 
must think not only about what is possible if the con- 
ditional is true: there are apples and oranges; they must 
also think about what is not possible if the conditional is 
true: there are apples but no oranges. Moreover, they do 
not think about their beliefs concerning what happens 
when there are no apples, because they consider such 
instances irrelevant (Evans et al., 2003). 

Recent empirical studies about judgments of truth 
and probability test predictions derived from these com- 
peting accounts (e.g., Byrne & Johnson-Laird, 2019; Cruz 
et al., 2015; Elqgayam & Over, 2013; Goodwin, 2014; 
Hinterecker, Knauff, & Johnson-Laird, 2016; Pfeifer & 
Tulkki, 2017). I will illustrate just one of the discoveries. 
Consider the following short narrative: 


Carmen went shopping to the market. When she looked at 
the poster there, she saw it said, “If there are apples, there 
are oranges.” When Carmen looked at the shelves, she saw 
that there were apples and there were oranges. Carmen 
checked her list of purchases. 


When participants in experiments read a conditional such 
as “If there are apples, there are oranges” in this short nar- 
rative, they are able to read the subsequent conjunction 
“There were apples and there were oranges” very quickly, 
in just under a second and a half, on average (Espino, San- 
tamaria, & Byrne, 2009). The conditional “primes” them 
to read the affirmative conjunction quickly. A bicondi- 
tional such as “If and only if there are apples, there are 
oranges” also primes them to read the affirmative con- 
junction just as quickly compared to the conditional. 
Now consider instead the following narrative: 


Carmen went shopping to the market. When she looked at 
the poster there, she saw it said, “If there are apples, there 
are oranges.” When Carmen looked at the shelves, she saw 
that there were no apples and there were oranges. Carmen 
checked her list of purchases. 


Participants read the conjunction with the negated 
antecedent—“There were no apples and there were 
oranges”—more quickly when they were primed by 
a conditional compared to a biconditional (see fig- 
ure 6.3.1). Why does the conditional prime participants 
to read the conjunction with the negative antecedent, 
“There were no apples and there were oranges,” more 
quickly compared to the biconditional? 

The mental model theory explains the experimental 
result readily: people mentally represent what is possible, 
not what is impossible. They read the conjunction “There 
were no apples and there were oranges” more quickly 
when it is primed by a conditional, because it is one of 
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the true possibilities consistent with the conditional, 
whereas they take longer to read the conjunction when it 
is primed by the biconditional, because it is a false possi- 
bility ruled out by the biconditional (Espino et al., 2009). 

The experimental result is more challenging to explain 
on the probabilist account. People do not think about their 
beliefs concerning what happens when the “if” clause is 
not true (Evans et al., 2003). So the theory predicts no 
difference in reading times for “There are no apples and 
there are oranges” when primed by a conditional versus 
a biconditional. In fact, the suppositional version even 
predicts a difference in the opposite direction. A bicondi- 
tional is understood as two related conditionals, “If there 
are apples, there are oranges, and if there are oranges, 
there are apples” (Evans et al., 2003; see also Handley, 
Evans, & Thompson, 2006). People must compare their 
belief that there are apples and there are oranges to their 
belief that there are apples and there are no oranges for 
the first conditional, and in addition, they must compare 
their belief that there are oranges and there are apples to 
their belief that there are oranges and there are no apples 
for the second conditional. Thus, for a biconditional, 
they think about three prior beliefs, rather than just two 
(see table 6.3.2). The theory must predict that people will 
read the conjunction “There are no apples and there are 
oranges” more quickly when it is primed by the bicondi- 
tional than the conditional, the opposite of the experi- 
mental result (Espino et al., 2009). 

We can conclude that when people understand a con- 
ditional, their initial thoughts are about what is possible, 
not what is impossible. We can also conclude that when 
they understand a conditional, their initial thoughts are 
not about what is probable (since that would require 
them to have thought about what is impossible as well 
as what is possible). 


2.2 When Do People Infer a Conditional Relation? 

Many studies examine how people infer a causal rela- 
tionship from the observation of objects and events (for 
reviews, see Johnson-Laird & Khemlani, 2017; Oaksford 
& Chater, 2017). But people can infer a conditional rela- 
tionship even in non-causal situations. Suppose you 
know “There are roses in the garden or there are lilies or 
both.” Can you infer “If there are no roses there are lil- 
ies”? About half of participants in experiments make the 
inference (Espino & Byrne, 2013). It occurs even when 
the assertions contain negated clauses, for example, 
when people are told “There are no daisies in the garden 
or there are no buttercups or both,” they infer “If there 
are daisies, there are no buttercups” (see figure 6.3.2). 
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Figure 6.3.1 


There were apples 
and there were oranges 


® Ifthere are apples, then there are oranges 


Hl If and only if there are apples, then there are oranges 


The discovery that when people hear about disjunctive 
alternatives, they make an inference of a conditional 
relation between them, is intriguing. Why would they 
do so? 

According to the mental model theory, the inference 
is a reasonable one to make (Johnson-Laird & Byrne, 
2002; see also Ormerod & Richardson, 2003). The dis- 
junction “There are roses or lilies or both” is consistent 
with three possibilities: 


roses no lilies 
no roses lilies 
roses lilies 


And the conditional “If there are no roses there are lil- 
ies” is consistent with the same three possibilities: 


no roses lilies 
roses no lilies 
roses lilies 


Because there are no possibilities that the two assertions 
do not have in common, the inference from the dis- 
junction to the conditional is valid—there are no coun- 
terexamples to it. Of course, the inference is difficult, 
requiring the comparison of multiple possibilities for 
each assertion, which is likely to exceed most people’s 
working-memory capacity (see also Murray & Byrne, 
2005). Unsurprisingly, only about half of participants 
make the inference (see figure 6.3.2). Instead, many of 
them make a revealing error: they choose the conclusion 


There were no apples 
and there were oranges 


Reading times for conjunctions primed 
by conditionals or biconditionals (adapted 
from Espino et al., 2009, experiment 2). 
Error bars are standard error of the mean. 


that matches their initial model of the conditional (see 
Espino & Byrne, 2013). 

Crucially, according to the mental model theory, the 
inference between the disjunction and the conditional is 
valid in either direction, from “or” to “if” and from “if” 
to “or.” For example, suppose you know instead “If there 
are no tulips in the garden, there are daffodils.” Can you 
infer “There are tulips or daffodils or both”? About one 
third of participants make the inference. Tellingly, in 
most cases, participants make the inference from “or” 
to “if” more often than the one from “if” to “or,” as fig- 
ure 6.3.2 shows. 

The difference is revealing because it challenges the 
predictions of probabilist theories (Oberauer, Geiger, & 
Fischer, 2011; Over, Evans, & Elqayam, 2010). Accord- 
ing to probabilist theories, the inference from “or” to 
“if’”—the one that people make most often—is not valid; 
only the one from “if” to “or” is valid. A probabilistically 
valid (p-valid) inference is one in which the likelihood 
of the conclusion is not lower than the likelihood of the 
premise. The inference from “or” to “if” is not p-valid 
because the likelihood of the conditional can be lower 
than the likelihood of the disjunction. The equation for 
the general relation between the probability of a disjunc- 
tion “A or B,” P(A or B), and the probability of a condi- 
tional “If not-A, B,” P(B | not-A), is as follows (as given in 
Over et al., 2010, p. 142): 


P(A or B) = P(A) +P(B | not-A) - P(A)P(B | not-A) 
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Hence, the inference from “or” to “if” is not p-valid 
(see also Oberauer et al., 2011). In contrast, the inference 
from “if” to “or” is p-valid because the likelihood of the 
disjunction cannot be smaller than the likelihood of the 
conditional. The relation is shown by the expansion of 
the probability of the conditional (as provided in Ober- 
auer et al., 2011, p. 97), given by P(B | not-A): 


P(A) x P(B | not-A) + P(not-A) x P(B | not-A) 
and the probability of the disjunction: 
P(A) + P(not-A) x P(B | not-A) 


People judge the probability of a conditional premise 
to be higher than, or the same as, the probability of a 
disjunctive conclusion for “if” to “or” inferences, and 
they judge the probability of a conditional conclusion to 
be higher than, or the same as, the probability of a dis- 
junctive premise for “or” to “if” inferences (Cruz et al., 
2015; see also Gilio & Over, 2012). But nonetheless, as 
figure 6.3.2 shows, they make the inference from “or” 
to “if” more often than the inference from “if” to “or,” 
even though the former is p-invalid and the latter is 
p-valid. The experimental result is the opposite of what 
would be expected if people understood conditionals by 
calculating likelihood, and it runs counter to the predic- 
tions of probabilist theories. 

We can conclude that people are willing to infer a 
conditional relation from alternative possibilities. They 
infer a conditional from a disjunction because they 
think about the possibilities that are consistent with 
each assertion and appreciate that they are the same. 
Although the inference is valid in both directions in 
that there are no counterexample possibilities, people 
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Figure 6.3.2 
AornoB Percentages of inferences from a disjunction 


to a conditional and from a conditional to a 


Ifno A then no B 


disjunction (adapted from Espino & Byrne, 


2013, experiment 1b). Error bars are stan- 
dard error of the mean. 


may be more inclined to make the inference from the 
disjunction to the conditional because the disjunction 
appears to assert categorically that something is the case, 
in contrast to the conditional, which appears to assert a 
hypothetical relation. 

These two key discoveries shed light on the nature of 
the mental algorithms that underlie conditional infer- 
ence. Further light is shed by discoveries about infer- 
ences from counterfactual conditionals. 


3. Counterfactual Reasoning 


A counterfactual such as “If she had done everything 
right, she would have passed her driving test” can seem 
to mean something very different from its factual coun- 
terpart, “If she did everything right, she passed her driv- 
ing test.” The counterfactual seems to convey that its 
antecedent is not the case—“She did not do everything 
right”—and it seems to convey that its consequent is 
not the case either—“She did not pass her driving test.” 
Hence, assessing the truth of a counterfactual poses 
unique logical problems—since every counterfactual has 
a false antecedent, every counterfactual must be true on 
a traditional truth-functional account of conditionals, 
according to which a conditional is true if its antecedent 
is false or its consequent is true (see Nickerson, 2015). 
The development of possible-worlds semantics in modal 
logics led to several alternative analyses of counterfac- 
tuals (e.g., Lewis, 1973; Stalnaker, 1968). Their study 
in the psychology of reasoning has also had a signifi- 
cant impact in the past 20 years since they have come 
under increasing scrutiny in experiments. Two research 
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questions, and the discoveries they led to, illustrate the 
novel contribution of the study of counterfactuals to 
understanding the reasoning mind. 


3.1 Do People Make Similar Inferences from Factual 
and Counterfactual Conditionals? 
From a conditional in the indicative mood—which is 
often called a factual conditional—such as, “If Paolo 
was in Venice, then Marco was in Padua,” when people 
are told, “Marco was not in Padua,” only about half of 
them make the modus tollens inference, “Paolo was not 
in Venice,” and the remainder tend to say that nothing 
follows. Strikingly, inferences from a conditional in the 
subjunctive mood—which is often called a counterfac- 
tual conditional—such as, “If Paolo had been in Venice, 
then Marco would have been in Padua,” show a different 
pattern. When people know “Marco was not in Padua,” 
most of them readily make the modus tollens inference 
“Paolo was not in Venice.” The otherwise complex infer- 
ence is made easily from the counterfactual (Byrne & 
Tasso 1999), as figure 6.3.3 shows. 

Moreover, people continue to make the modus ponens 
inference, from “Paolo was in Venice” to “Marco was in 
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Padua,” as readily from the counterfactual as from the 
factual conditional. Similar effects occur for the denial of 
the antecedent inference, which is made more often from 
the counterfactual than the factual conditional, and the 
affirmation of the consequent inference, which is made 
equally from both sorts of conditionals. 

This “counterfactual inference effect” is robust and 
has been studied extensively (for a review, see Byrne, 
2017). It occurs not only for locational content but also 
for causal and definitional counterfactuals (Thompson 
& Byrne, 2002) and various everyday scenarios (Frosch 
& Byrne, 2012). It has been examined for inducements 
such as promises and threats (Egan & Byrne, 2012) and 
deontic content such as obligations (Quelhas & Byrne, 
2003). Why are otherwise difficult inferences easy from 
a counterfactual? 

The discovery helps distinguish alternative explana- 
tions of human reasoning. According to the mental model 
theory, people understand a counterfactual such as “If 
there had been apples there would have been oranges” 
by thinking about two possibilities from the outset. They 
mentally represent the conjecture mentioned in the con- 
ditional, “There are apples and there are oranges,” and 


®@ Factual 


Figure 6.3.3 


Locational Causal Definitional Everyday Promises Threats Deontic 
Byrne & Tasso, | Thompson & Byrne, 2002, Exp. 3 | Frosch & Byrne, Egan & Byrne, 2012, Exp. 2 Quelhas & Byrne, 
1999, Exp. 3 2012, Exp. 3a 2003, Exp. 1 


Hi Counterfactual 


The percentages of modus tollens inferences made from factual and counterfactual conditionals for different contents (adapted 
from Byrne & Tasso, 1999; Egan & Byrne, 2012; Frosch & Byrne, 2012; Quelhas & Byrne, 2003; Thompson & Byrne, 2002). 
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they also mentally represent the presupposed or known 
facts, “There are no apples and there are no oranges.” 
They keep track of the epistemic status of their models, as 
corresponding to a counterfactual possibility, or the facts 
(e.g., Johnson-Laird & Byrne, 1991): 


Counterfactual: 
Facts: 


apples 
no apples 


oranges 
no oranges 


The theory explains a number of curious findings 
about counterfactuals. For example, when people hear 
the counterfactual, they mistakenly remember that they 
were told there were no apples and no oranges (Fillen- 
baum, 1974). They judge that someone who utters the 
counterfactual means to imply there were no apples and 
no oranges (Thompson & Byrne, 2002). Strikingly, they 
read the conjunction “There were no apples and there 
were no oranges” more quickly when they are primed 
by the counterfactual compared to the factual condi- 
tional, but they read the conjunction “There were apples 
and there were oranges” equally quickly when they are 
primed by either conditional, as figure 6.3.4 shows (San- 
tamaria, Espino, & Byrne, 2005; see also de Vega, Urrutia, 
& Riffo, 2007). They look at images corresponding to “no 
apples and no oranges” more often for the counterfactual 
than the factual conditional, as shown by eye-tracking 
studies (Orenes, Garcia-Madruga, GOmez-Veiga, Espino, 
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& Byrne, 2019; see also Ferguson & Sanford, 2008; Nieuw- 
land & Martin, 2012). The dual meaning of counterfactu- 
als has been corroborated by studies that rely on measures 
of brain activity such as event-related potential (ERP) 
measures and functional magnetic resonance imaging 
(fMRI) measures (e.g., Ferguson, Sanford, & Leuthold, 
2008; Kulakova, Aichhorn, Schurz, Kronbichler, & Perner, 
2013; Van Hoeck et al., 2013). 

The proposal that people think of two possibilities 
from the outset for the counterfactual predicts the find- 
ing that they readily make the modus tollens inference 
(Byrne & Tasso, 1999). When they hear, “There are no 
oranges,” they do not have to flesh out their models, 
as they do for a factual conditional. They have already 
represented a possibility in which there are no oranges. 
They can eliminate the model corresponding to the 
counterfactual conjecture and conclude, “There are no 
apples.” Similarly, they make the modus ponens infer- 
ence as readily from the counterfactual as the factual 
conditional because when they hear, “There are apples,” 
they can update their models to eliminate the model 
corresponding to the presupposed facts and note instead 
that the facts are that there are apples and oranges. They 
can then conclude, “There are oranges.” 

Of course, the fully fleshed out models for the coun- 
terfactual are the same as those for the factual condi- 
tional, even though the epistemic status of the models 
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There were apples and 
there were oranges 


There were no apples and 
there were no oranges 


® If there were apples, there were oranges 


If there had been apples, there would have been oranges 


Figure 6.3.4 


Reading times for conjunctions primed by factual or counterfactual conditionals (adapted from Santamaria et al., 2005, experi- 


ment 1). Error bars are standard error of the mean. 
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is different, for example, on a conditional interpretation 
of “if”: 


Counterfactual: apples oranges 
Facts: no apples no oranges 
Counterfactual: no apples oranges 


Even though counterfactuals can seem to mean some- 
thing very different from factual conditionals, their 
semantics runs in parallel (e.g., Byrne & Johnson-Laird, 
2019). A counterfactual such as “If Oswald hadn’t shot 
Kennedy, then someone else would have” seems debat- 
able, whereas the corresponding factual one, “If Oswald 
didn’t shoot Kennedy, then someone else did,” seems 
true (Adams, 1970). But the comparison is misleading 
(Byrne & Johnson-Laird, 2019). The parallel to Adams’s 
counterfactual should be “If Oswald hasn’t shot Ken- 
nedy, then someone else will,” which is also debatable. 
And the counterfactual parallel to his factual condi- 
tional, which assumes Kennedy has been shot, should 
be, “If Oswald hadn’t been the person who shot Ken- 
nedy, then someone else must have been,” which indeed 
seems true (see Byrne & Johnson-Laird, 2019). 

One probabilist approach proposes that people 
understand a counterfactual by assessing its likelihood, 
which at the present time is the same as the probabil- 
ity of the factual conditional at a previous time (Over 
et al., 2007). People think only about one situation, in 
line with a singularity principle (Evans, 2007, p. 74). 
They combine their prior belief in the conditional prob- 
ability of the counterfactual, that is, the likelihood of 
the counterfactual consequent “There would have been 
oranges” given the counterfactual antecedent “if there 
had been apples,” with their prior beliefs in each of the 
implied facts, “There were no apples” and “There were 
no oranges,” to arrive at a single assessment of their 
belief in the counterfactual (Over et al., 2007). Hence, 
it is challenging for this suppositional theory to explain 
the findings that people make modus tollens more often 
from counterfactuals than factual conditionals and that 
they appear to have ready access not only to the coun- 
terfactual conjecture but also to the presupposed facts. 

Another probabilist theory is based on causal Bayes 
nets (e.g., Pearl, 2013). People construct causal models, 
for example, 


OBO 


in which nodes represent causes and effects, arrows 
indicate causal direction, and a conditional probability 
table at each node gives the probability that a node is 
present or absent (e.g., Lucas & Kemp, 2015; Sloman 
& Lagnado, 2005). Changes to the causal model are 
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made by counterfactual “interventions” on a node (e.g., 
imagining that B is absent deletes links into B). A coun- 
terfactual’s probability relies on the “do” intervention 
operator, P(c| b. do(-b)), that is, the conditional prob- 
ability of c, given that b was observed, but counterfactu- 
ally removed (e.g., Dehghani, Iliev, & Kaufmann, 2012; 
Lagnado, Gerstenberg, & Zultan, 2013; Meder, Hag- 
mayer, & Waldmann, 2009). The account predicts the 
opposite of the counterfactual inference effect, namely, 
that people refrain from modus tollens when counter- 
factuals describe the prevention of a cause (e.g., Sloman 
& Lagnado, 2005). For example, participants tend to 
make the modus tollens inference for a factual ques- 
tion, “Suppose A was observed to not be moving, would 
B still be moving?” but not for a counterfactual one, 
“Suppose A were prevented from moving, would B still 
be moving?” (Sloman & Lagnado, 2005). A challenge 
for the causal model theory is that the counterfactual 
inference effect is observed for conditionals that convey 
not only causal relations but also a wide variety of other 
sorts of conditional relation, as figure 6.3.3 shows. 

Of course, not all conditionals in the subjunctive 
mood are counterfactuals (e.g., Dudman, 1988). So-called 
Anderson conditionals (e.g., “If he had taken arsenic, 
he would have exactly these symptoms”; Adams, 1975) 
make use of the subjunctive mood, but their antecedents 
may be indicatives in disguise (e.g., “If he in fact took 
arsenic, he would have exactly these symptoms”). And 
other counterfactuals, such as counterpossibles (coun- 
terfactuals with impossible antecedents), for example, 
“If Hobbes had squared the circle, sick children in the 
mountains of South America at the time would have 
cared” (see Berto & Jago, 2018; Williamson, 2018), have 
received scant psychological attention. 


3.2 Do People Construct “Embodied” Mental 
Representations of Counterfactuals? 
There is abundant evidence that when people understand 
a counterfactual such as “If the flowers had been roses, the 
trees would have been orange trees,” they think about two 
possibilities, the conjecture, “There were roses and orange 
trees,” and its opposite, the presupposed facts. But how are 
the presupposed facts mentally represented? They could 
be represented explicitly through the use of negation, 
“There are no roses and there are no orange trees,” or 
they could be represented by alternates, “There are pop- 
pies and there are apple trees.” Which sorts of mental 
representations do people rely on? 

The idea that the presupposed facts are represented as 
alternates is derived from the theory that the meaning of 
concepts, such as “roses,” is embodied, ina modality-specific, 
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experientially based representation grounded in character- 
istics of sensory or motor processes (e.g., Barsalou, Sim- 
mons, Barbey, & Wilson, 2003). The embodied proposal 
extends even to abstract concepts such as negation (e.g., 
Glenberg, Robertson, Jansen, & Johnson-Glenberg, 1999). 
In an embodied system, a negation such as “There is not 
a rose” is represented by an alternate, such as “There is 
a poppy,” and it may require several steps, for example, 
first representing “There is a rose” and then inhibiting the 
representation (e.g., Kaup, Liidtke, & Zwaan, 2006; Mayo, 
Schul, & Burnstein, 2004). 

In an experimental test of the embodied idea, partici- 
pants were told that in a garden, the flowers are roses or 
poppies and the trees are orange trees or apple trees, and 
they then read a counterfactual, “If the flowers had been 
roses, the trees would have been orange trees.” When 
they were told, “The trees were not orange trees,” most 
of them tended to infer, “The flowers were poppies” 
(Espino & Byrne, 2018). They rarely said, “The flowers 
were not roses.” In other words, they tended to infer 
what is the case, from information about what is not 
the case. This “inference-to-alternates effect” is a robust 
strategy that occurs in many different situations. 

One situation in which the inference-to-alternates 
effect does not occur is revealing about the source of 
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the tendency. Participants were told that the flowers are 
roses or poppies or lilies and the trees are orange trees or 
apple trees, that is, the context for flowers was a multiple 
one rather than a binary one. Now when they read a 
counterfactual “If the flowers had been roses, the trees 
would have been orange trees,” and “The trees were not 
orange trees,” many of them said, “The flowers were not 
roses,” as figure 6.3.5 shows. 

Why do people exhibit an inference-to-alternates 
effect in a binary context but not in a multiple con- 
text? On the embodied view, the presupposed facts 
for the counterfactual are represented as alternates, in 
both sorts of context. It explains the observation of an 
inference-to-alternates effect in the binary context, but 
it makes the wrong prediction for the multiple context. 
When people are told, “The trees are not orange trees,” 
they should conclude, “There are poppies or lilies.” But 
they do not do so. 

According to the mental model theory, the mental 
representation of negation is as iconic as possible, but 
it can include symbols, such as a propositional-like tag 
“no” or some other annotation to capture negation (e.g., 
Johnson-Laird & Byrne, 2002; Orenes, Beltran, & Santa- 
maria, 2014). In a binary context, people simulate the 
presupposed facts by thinking about alternates: 


Experiment | Binary 


Bf Affirmative 


Figure 6.3.5 
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Percentages of modus ponens (MP), denial of the antecedent (DA), affirmation of the consequent (AC), and modus tollens (MT) 


inferences from counterfactuals in a binary or multiple context phrased as affirmative, e.g., “The flowers were poppies,” or as nega- 


tive, e.g., “The flowers were not roses” (adapted from Espino & Byrne, 2018). Error bars are standard error of the mean. 
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roses 
poppies 


Counterfactual: 
Facts: 


oranges 
apples 


When they are told, “There are no oranges,” they 
eliminate the first model and conclude, based on the sec- 
ond model, “There are poppies.” Hence, the inference- 
to-alternates effect is observed. But in a multiple context, 
the number of alternates they would have to consider is 
likely to exceed working memory: 


Counterfactual: roses oranges 
Facts: poppies apples 
lilies apples 


Instead they can switch to models that are annotated 
with propositional-like tags or symbols: 


Counterfactual: 
Facts: 


TOSes 
no roses 


oranges 
no oranges 


Hence, when they are told, “There are no oranges,” 
they conclude, based on the second model, “There are 
no roses,” that is, the inference-to-alternates effect is 
eliminated. We can conclude that how people repre- 
sent the presupposed facts depends on the context. 
People tend to mentally simulate the presupposed facts 
by considering alternates but they can switch strategy 
to a symbolic annotation when the possibilities exceed 
working-memory constraints. 

These two key discoveries about inferences from coun- 
terfactuals shed further light on the nature of the mental 
algorithms that underlie conditional inference. 


4. Conclusion 


Conditional inferences are at the very center of everyday 
human reasoning, and counterfactual inferences occupy 
a special place there. Conditionals and counterfactuals 
provide a window onto the nature of mental simulations 
and algorithms in the human mind. In this chapter, we 
have considered some important discoveries that help 
distinguish explanations of conditional and counterfac- 
tual reasoning. When people understand, and reason 
about, a factual conditional, they think about what the 
conditional indicates is possible, not what it rules out as 
impossible. When they hear about alternative possibili- 
ties, they infer a conditional relation between them. The 
findings indicate that people think about what is possi- 
ble, not what is probable. When they understand, and 
reason about, a counterfactual conditional, they are able 
to make complex inferences very readily. They make 
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inferences about what is the case, even when they are told 
what is not the case. The findings indicate that people 
think about dual possibilities for a counterfactual—not 
only the counterfactual conjecture but also the presup- 
posed facts—and they construct iconic mental simula- 
tions that can include symbolic annotations. These 
discoveries about conditional and counterfactual rea- 
soning provide important insights into how the mind 
accomplishes reasoning by imagining possibilities of dif- 
ferent sorts. 
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6.4 Utility Conditionals 


Jean-Francois Bonnefon 


Summary 


Utility conditionals are statements like “If you update 
this app, your phone will crash” or “If I let you do this, 
my boss will fire me,” where antecedents, consequents, 
or both are actions or outcomes that are desirable or 
undesirable for some agents. Because they describe desir- 
able and undesirable states of the world, utility condi- 
tionals support peculiar inferences that draw on both 
theoretical rationality and practical rationality. They 
offer a convenient device for studying how people reason 
about the beliefs, preferences, and decisions of others. 
As a consequence, they are a natural point of contact 
between disparate fields such as theory of mind, Bayes- 
ian approaches to reasoning, and behavioral economics. 


1. Three Ways We Reason with Utility Conditionals 


Utility conditionals (Bonnefon, 2009) are statements of 
the form “if p, q” where p or q (or both) are actions that 
increase or decrease the utility (in the economic sense) 
of some agents. For example: 
(1) a. If you wash the car, I’ll let you borrow it tonight. 
. If you update this app, your phone will crash. 

If you testify against me, you will have an accident. 
. If 1 let you do this, my boss will fire me. 


. If Peyton moves to Paris, Alex will be unhappy. 


moan eT 


If I study instead of partying, I will get good grades. 


Your washing the car is useful to me, you would not like 
your phone to crash, having accidents is bad, I don’t 
want to be fired, Alex would be unhappy to be unhappy, 
and having good grades is good. Thus, all these exam- 
ples are utility conditionals. 

Some utility conditionals are speech acts, statements 
that perform a social function, for which we have a 
label: conditional (1a) is a promise, and conditional (1b) 
is a warning. Other utility conditionals are speech acts 
for which we do not have as clear a label: conditional 


(1c) would be something like a veiled threat, and condi- 
tional (1d) is something like an apologetic interdiction. 
Other utility conditionals yet, such as (le) and (1f), do 
not seem to be speech acts at all. 

One line of research about utility conditionals is con- 
cerned with how people sort them out into categories 
such as promises, threats, tips, and warnings (these cate- 
gories are also of interest for deontic reasoning; see chap- 
ter 11.2 by Elqayam, this handbook). For example, in a 
promise such as conditional (1a), the consequent of the 
conditional is an event under the control of the speaker, 
which has positive valence for the listener. Change the 
valence to something negative, and you get a threat; 
change the event such that it is no longer under the con- 
trol of the speaker, and you get a tip; change both, as in 
conditional (1b), and you get a warning (Evans, 2005; 
Lopez-Rousseau & Ketelaar, 2004, 2006). 

The frontiers between these four categories can be 
blurry, though. For example, conditional (1c) has the 
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Figure 6.4.1 


Distance between promises, threats, tips, and warnings along 
the axes of valence and control. Some “warnings” are really 
veiled threats, and some threats are issued as “promises,” per- 
haps for dramatic effect. 
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properties of a warning but sounds like a threat. And 
indeed, eye-tracking studies have shown that readers are 
not too perturbed when such a statement is referred to as a 
threat (Wray, Wood, Haigh, & Stewart, 2016)—suggesting 
that the speaker’s explicit control over the realization of 
q is not a strong constraint for distinguishing between 
threats and warnings. Altogether, other eye-tracking stud- 
ies (Haigh, Ferguson, & Stewart, 2014; Haigh, Stewart, 
Wood, & Connell, 2011; Stewart, Haigh, & Ferguson, 
2013; Wood, Haigh, & Stewart, 2016) paint a picture 
not unlike figure 6.4.1: the frontiers around threats are 
blurry (both warnings and promises can be disguised or 
“indirect” threats), but tips are clearly identifiable as con- 
ditionals “if p, q” where qg is something positive for the 
listener over which the speaker has no control. 

Regardless of the way we sort utility conditionals into 
linguistic or pragmatic categories, there are three ways to 
reason about them, which involve three different forms 
or mixtures of rationality. 

The first way, which I will not address in this chap- 
ter because it only concerns a subset of utility con- 
ditionals, is to consider their use as persuasion tools. 
Indeed, some utility conditionals can be used as argu- 
ments for or against a given course of action or policy 
(Schellens & De Jong, 2004). More precisely, these 
utility conditionals are equivalent to arguments from 
consequences, in which the good or bad consequences 
of some action are pointed out in order to argue that 
this action should or should not be taken (Bonnefon, 
2016). The persuasive use of utility conditionals appeals 
to practical rationality. Indeed, a core question within 
this field of research is to assess whether people are 
convinced by “good” arguments—arguments that would 
convince a rational agent who wants actions to maxi- 
mize expected utility (Corner & Hahn, 2007; Corner, 
Hahn, & Oaksford, 2011; Feteris, 2002; Hahn & Oaks- 
ford, 2006, 2007; Hoeken, Sorm, & Schellens, 2014; 
Hoeken, Timmers, & Schellens, 2012; Thompson, Evans, 
& Handley, 2005). 

The second way one can reason about utility condi- 
tionals is to use them to derive standard syllogistic infer- 
ences (modus ponens, modus tollens, etc.), for example, 


(2) a. If Peyton moves to Paris, Alex will be unhappy. 
b. Alex is happy. 
c. Therefore, Peyton has not moved to Paris. 
This form of reasoning about utility conditionals primar- 
ily draws on theoretical rationality, that is, it mostly aims 
to maintain belief consistency. We will see, though, that 


practical rationality can sneak into this kind of reason- 
ing, because what we believe of the practical rationality 
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of others can affect the confidence we have in utility con- 
ditionals and thus the confidence we have in the conclu- 
sions of syllogisms that use them as main premises. For 
example, consider 


(3) a. Lucy, if you don’t eat your salad, you will be 
grounded for 5 years. 


b. Lucy does not eat her salad. 


c. Therefore, Lucy is grounded for 5 years. 


As I will discuss in section 2, the reason why this con- 
clusion sounds doubtful has to do with the lack of cred- 
ibility of conditional (3a), and this lack of credibility 
derives from our assumptions about whether it would 
be practically rational to enforce the threat expressed in 
conditional (3a). 

The third way one can reason about utility condi- 
tionals is to use them to derive nonstandard inferences 
about what agents want, believe, and do. For example: 


(4) a. If Alex leaves early, she will be fired. 


b. Therefore, Alex won’t leave early. 


Icall this inference “nonstandard” because it is not licensed 
by a logical reading of (4a). Indeed, there are no logical 
grounds to infer not-p from “if p, q.” Instead, the inference 
that Alex won’t leave early is based on the assumption that 
Alex is a practically rational agent who would not want to 
be fired and thus will not take an action that would get her 
fired. As I will discuss in section 3, such nonstandard infer- 
ences can concern actions, as in (4b), but also preferences 
and beliefs. 


2. Standard Inferences 


For the sake of simplicity, I will focus in this section on 
the most basic conditional inference, modus ponens (for 
other phenomena, see, e.g., Couto, Quelhas, & Byrne, 
2017; Demeure, Bonnefon, & Raufaste, 2009; Hilton, 
Kemmelmeier, & Bonnefon, 2005). Given a utility con- 
ditional “if p, q” and the observation that p is true, how 
likely are we to conclude that q is true? 

First, we need to observe that some utility condition- 
als are more credible than others. For example, I have 
already discussed the distinction between promises and 
tips. When speakers make promises, they usually are ina 
position to fulfill these promises: 


(5) If you tidy up your room, I’ll make your favorite 
dessert. 


But when speakers give tips, they cannot actually guar- 
antee that the tip will actually result in the expected 
consequence: 
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(6) If you work hard, your teacher will give you a good 
grade. 


Accordingly, we can expect reasoners to be relatively 
confident that promises will be carried out but less con- 
fident in the consequences of tips. As a consequence, 
reasoners are more likely to carry out modus ponens 
inferences from promises than from tips (Couto et al., 
2017; Evans, Neilens, Handley, & Over, 2008), and muta- 
tis mutandis, they are more likely to carry out modus 
ponens inferences from threats than from warnings. 

Even promises and threats, though, can be more or 
less credible. Compare, for example, the following two 
conditionals: 


(7) a. If you don’t eat your salad, you will be grounded 
tonight. 


b. If you don’t eat your salad, you will be grounded 
for 5 years. 


Threat (7b) is so disproportionate to the offense that it 
loses its credibility (Amgoud, Bonnefon, & Prade, 2005; 
Lopez-Rousseau, Diesendruck, & Benozio, 2011; Ver- 
brugge, Dieussaert, Schaeken, & Van Belle, 2004). Par- 
ents who would unleash such punishment onto their 
children for mere peccadilloes would soon find them- 
selves out of options—not to mention the effort that 
would be required from the parents to actually maintain 
the punishment for 5 years. Assuming that the parents 
have some modicum of practical rationality, it seems 
unlikely that they would actually carry out the threat. 

Just like the vast majority of conditionals, utility con- 
ditionals have exceptions, that is, circumstances that 
weaken the inferential connection from their anteced- 
ent to their consequent. For example, Gazzo Castaneda 
and Knauff (2016b) have investigated “if crime, then 
punishment” utility conditionals, such as 


(8) a. If a person downloads music from the Internet 
without allowance, then this person will be pun- 
ished for breaching the copyright law. 

b. If a person kills another human, then this person 
will be punished for manslaughter. 


Because reasoners find it easier to imagine exceptions to 
(8a) than to (8b), they are less likely to derive modus 
ponens inferences from (8a) than from (8b). In other 
words, utility conditionals behave just like regular con- 
ditionals in this respect. However, the magnitude of the 
crime (i.e., the disutility inflicted by whomever took the 
action in the antecedent clause) affects modus ponens 
performance when the modal “will” is replaced with the 
modal “should” (Gazzo Castafieda & Knauff, 2016a): 
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(9) a. If a person gains admission to an event without 
paying, then this person should be punished for 
obtaining benefits by devious means. 


b. Ifa person maltreats a minor in their charge, then 
this person should be punished for maltreatment 
of wards. 


Reasoners take into account exceptions and exculpatory 
circumstances when they apply modus ponens to (9a), 
but they ignore these exceptions and exculpatory circum- 
stances when dealing with (9b), because of the moral out- 
rage they feel about maltreatment. In other words, the 
utility component of the conditional moderates the effect 
of exceptions on the application of modus ponens (see 
also Gazzo Castafieda, Richter, & Knauff, 2016). 


3. Nonstandard Inferences 


The most important feature of utility conditionals is 
that they support nonstandard inferences that are based 
on assumptions of practical rationality. Consider this 
example: 


(10) If Alex writes an application, she’ll get the award. 


Conditional (10) suggests that Alex will write an appli- 
cation, based on three assumptions: that she wants the 
award enough to take the time to write an application, 
that she knows she’ll get the award if she does, and that 
she’s a rational agent who takes actions that increase her 
net utility. People are very well attuned to this calculus 
(Bonnefon & Hilton, 2004; Bonnefon & Sloman, 2013; 
Elgayam, Thompson, Wilkinson, Evans, & Over, 2015; 
Evans et al., 2008; Jara-Ettinger, Hyowon, Schulz, & 
Tenenbaum, 2016). Given a utility conditional “if p, q” 
where p is an action of an agent and q a positive (or neg- 
ative, respectively) consequence of that action, reasoners 
will infer that the agent will or will not (and even should 
or should not) do q. They are more likely to make that 
inference when the magnitude of g increases, they are 
less likely to make that inference when the probability of 
q given p decreases, and they are less likely to make that 
inference when they can think of a way to obtain q that 
is less costly than p. Furthermore, eye-tracking evidence 
suggests that they make these inferences very quickly, in 
real time, as they read the conditional (Haigh & Bonne- 
fon, 2015a, 2015b). 

Not only can people predict actions based on utility 
conditionals, but they can also make inferences about 
the beliefs and preferences of other agents: 


(11) a. If Brenda uses her loyalty card, she will save $20. 
b. Brenda does not use her loyalty card. 
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Conditional (11a) would usually encourage the infer- 
ence that Brenda uses her loyalty card. Given that this 
inference is falsified by (11b), reasoners look for a rea- 
son to explain the behavior of Brenda. Their expecta- 
tion that Brenda would use the loyalty card was based on 
at least four assumptions: (i) Brenda cares about saving 
$20, (ii) she knows she can save the $20 by using her loy- 
alty card, (iii) she’s a rational agent, and (iv) she is able 
to use her loyalty card. When they have no reason to 
believe that (iii) and (iv) may be the case, they decide to 
revise their assumptions about Brenda’s beliefs or prefer- 
ences: that is, they conclude either that Brenda did not 
know about the opportunity to save money or that she 
could not be bothered to save $20 (Bonnefon, Girotto, 
& Legrenzi, 2012). 

Utility conditionals can express more complex situa- 
tions than just one action and its consequences for the 
agent that takes it. Consider, for example, 


(12) If you hurt me, I will help her. 


This conditional encapsulates a complex, three-agent 
situation: if a hurts b, b will help c. Is a going to hurt b? 
Reasoners try to make sense of such a complex situation 
by fitting it into an easier template, such as a social con- 
tract in which people trade favors or exchange threats 
(Bonnefon, 2012; Bonnefon, Haigh, & Stewart, 2013; see 
also chapter 10.6 by Cosmides & Tooby, this handbook). 
More precisely, they hesitate between a good-for-good 
interpretation (for some reason, b wants to be hurt by 
a, and b wants a to help c; therefore, a will hurt b) anda 
bad-for-bad interpretation (a does not want b to help c, 
so b is threatening to help c if a hurts b; therefore, a will 
not hurt b). 

Reasoners make similar template-fitting inferences 
about the preferences of agents when they do not have 
intuitions about these preferences. Consider the follow- 
ing example, where “tymping” and “yorbing” are non- 
verbs whose interpretation is open: 


(13) If Peyton tymps Jesse, then Jesse will yorb Peyton. 


When told that Jesse enjoys being tymped, reasoners 
apply a social-contract template and infer that Peyton 
likes to be yorbed (and mutatis mutandis if told that 
Jesse does not enjoy being yorbed). Reasoners can even 
revise their interpretation of nonambiguous verbs such 
as “hurt”: 


(14) If Peyton hurts Jesse, then Jesse will zim Peyton. 


Here reasoners spontaneously assume that Peyton would 
not like to be zimmed. But if told that Peyton would 
actually like to be zimmed, they revise their assumptions 
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about the preferences of Jesse, coming to the conclu- 
sion that Jesse likes to be hurt (for other such template 
effects, see Bonnefon et al., 2013). 


4. Future Directions 


Utility conditionals bridge a gap between research on 
reasoning and research on decision making, between 
theoretical rationality and practical rationality. They 
offer a convenient device for studying how people 
reason about the beliefs, preferences, and decisions of 
others. As a consequence, they are a natural point of 
contact between disparate fields such as theory of mind, 
Bayesian approaches to reasoning, moral judgment (see 
chapter 12.3 by Wiegmann & Sauer, this handbook), and 
behavioral economics (De Vito & Bonnefon, 2014; Jara- 
Ettinger et al., 2016). The greater challenge ahead will be 
to tackle what economists call “other-regarding prefer- 
ences,” that is, preferences over the outcomes of other 
individuals (Cooper & Kagel, 2015). There is currently 
no well-defined theory about how people weight the 
utilities of others when making their decisions and least 
of all a theory of how people think others weight the 
utilities of others when making their decisions. Utility 
conditionals offer a useful tool to explore this uncharted 
territory and to discover the naive axioms of reasoners’ 
folk theory of decision. 
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Section 7 Causal and Diagnostic Reasoning 


7.1 Causal and Counterfactual Inference 


Judea Pearl 


Summary 


All accounts of rationality presuppose knowledge of how 
actions affect the state of the world and how the world 
would have changed had alternative actions been taken. 
The chapter presents a framework called the structural 
causal model (SCM), which operationalizes this knowl- 
edge and explicates how it can be derived from both theo- 
ries and data. In particular, it shows how counterfactuals 
are computed and how they can be embedded in a calcu- 
lus that solves critical problems in the empirical sciences. 


1. Actions, Physical and Metaphysical 


If the options available to an agent are specified in terms 
of their immediate consequences, as in “make him 
laugh,” “paint the wall red,” “raise taxes,” or, in general, 
do(X = x), where x is a value that variable X can take, then 
a rational agent is instructed to maximize the expected 
utility over all options x, 


EU(x) =} P.(y)U(y). (1) 
y 


Here, U(y) stands for the utility of outcome Y = y, and 
P,(y)—the focus of this chapter—stands for the (subjec- 
tive) probability that outcome Y = y would prevail had 
action do(X = x) been performed so as to establish condi- 
tion X = x. 

It has long been recognized that Bayesian conditional- 
ization, that is, P,(y) = P(y|x), is inappropriate for serving 
in equation (1), for it leads to paradoxical results of sev- 
eral kinds (see Pearl, 2000a, pp. 108-109; Skyrms, 1980). 
For example, patients would avoid going to the doc- 
tor to reduce the probability that they are seriously ill, 
barometers would be manipulated to reduce the chance 
of storms, doctors would recommend a drug to male 
and female patients but not to patients with undisclosed 
gender, and so on. Yet the question of what function 
should substitute for P,(y), despite decades of thoughtful 
debates (Cartwright, 1983; Harper, Stalnaker, & Pearce, 


1981; Jeffrey, 1965), seems to still baffle philosophers in 
the 21st century (Arl6-Costa, 2007; Weirich, 2008/2016; 
Woodward, 2003). Modern discussions over evidential 
versus causal decision theory (chapter 8.2 by Peterson, 
this handbook) echo these debates. 

Most studies of rationality have dealt with the util- 
ity function U(y), its behavior under various shades of 
uncertainty, and the adequacy of the expectation opera- 
tor in equation (1). Relatively little has been said about 
the probability P,(y) that governs outcomes Y = y when 
an action do(X = x) is contemplated. Yet regardless of what 
criterion one adopts for rational behavior, it must incor- 
porate knowledge of how our actions affect the world. 
We must therefore define the function P,(y) and explicate 
the process by which it is assessed or inferred, be it from 
empirical data or from world knowledge. We must also ask 
what mental representation and thought processes would 
permit a rational agent to combine world knowledge with 
empirical observations and compute P,(y). 

Guided by ideas from structural econometrics 
(Haavelmo, 1943; Spirtes, Glymour, & Scheines, 1993; 
Strotz & Wold, 1960), I have explored a conditioning 
operator called do(x) (Pearl, 1995) that captures the intent 
of P,(y) by simulating an intervention in a causal model 
of interdependent variables (Pearl, 2009b). 

The idea is simple. To model an action do(X = x), one 
performs a “mini-surgery” on the causal model, that is, 
a minimal change necessary for establishing the ante- 
cedent X = x, while leaving the rest of the model intact. 
This calls for removing the mechanism (i.e., equation) 
that nominally assigns values to variable X and replac- 
ing it with a new equation, X =x, that enforces the 
intent of the specified action. This mini-surgery (not 
unlike Lewis’s “little miracle”) makes precise the idea 
of using a “minimal deviation from actuality” to define 
counterfactuals. 

One important feature of this formulation is that the 
postintervention probability, P(y | do(x)), can be derived 
from preinterventional probabilities provided one pos- 
sesses a diagrammatic representation of the processes 
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that govern variables in the domain (Pearl, 2000a; 
Spirtes, Glymour, & Scheines, 2001). Specifically, the post- 
intervention probability reads:! 


P(x, y,z)/P(x|z), if x= x%, 
0, if x#x", 


P(x, y,z|do(X=x"*)) = (2) 
Here, z stands for any realization of the set Z of “past” 
variables, y is any realization of the set Y of “future” vari- 
ables, and “past” and “future” refer to the occurrence of 
the action event X = x*.? 

This feature, to be further discussed in section 2, is 
perhaps the key for the popularity of graph-theoretical 
methods in causal inference applications. It states that 
the effects of policies and interventions can be predicted 
without knowledge of the functional relationships (or 
mechanisms) among X, Y, and Z. The preinterventional 
probability and a few qualitative features of the model 
(e.g., variable ordering) are sufficient for determining 
the postintervention probabilities as in equation (2). 

The philosophical literature spawned a totally dif- 
ferent perspective on the probability function P,(y) in 
equation (1). In a famous letter to David Lewis, Robert 
Stalnaker (1972/1981) suggested to replace conditional 
probabilities with probabilities of conditionals, that is, 
P,(y) = P(x>y), where “x > y” stands for the counterfactual 
conditional “Y would be y if X were x” (see chapter 6.1 by 
Starr, this handbook). Using a “closest-worlds” semantics, 
Lewis (1973) defined P(x > y) using a probability-revision 
operation called “imaging,” in which probability mass 
“shifts” from worlds to worlds, governed by a measure 
of “similarity.” Whereas Bayes conditioning P(y|x) trans- 
fers the entire probability mass from worlds excluded by 
X =x to all remaining worlds, in proportion to the lat- 
ter’s prior probabilities P(-), imaging works differently: 
each excluded world w transfers its mass individually 
to a select set S,(w) of worlds that are considered “clos- 
est” to w among those satisfying X = x. Joyce (1999) used 
the “\”-symbol, as in “P(y\x),” to denote the probability 
resulting from such an imaging process and derived a for- 
mula for P(y \ x) in terms of the selection function S,(w). 

In Pearl (2000a, p. 73), I have shown that the trans- 
formation defined by the do-operator, equation (2), can 
be interpreted as an imaging-type mass transfer if the 
following two provisions are met: 


Provision 1: The choice of “similarity” measure is not 
arbitrary; worlds with equal histories should be con- 
sidered equally similar to any given world. 


Provision 2: The redistribution of weight within each 
selection set S,(w) is not arbitrary either; equally simi- 
lar worlds should receive mass in proportion to their 
prior probabilities. 


Judea Pearl 


This tie-breaking rule is similar in spirit to the Bayesian 
policy and permits us to generalize equation (2) to dis- 
junctive actions, as in “exercise at least 30 minutes daily” 
or “paint the wall either green or purple” (Pearl, 2017). 

The theory that emerges from the do-operator 
(equation (2)) offers several conceptual and operational 
advantages over Lewis’s closest-world semantics. First, it 
does not rest on a metaphysical notion of “similarity,” 
which may be different from person to person and thus 
could not explain the uniformity with which people 
interpret causal utterances. Instead, causal relations are 
defined in terms of our scientific understanding of how 
variables interact with one another (to be explicated in 
section 2). Second, it offers a plausible resolution of the 
“mental representation” puzzle: how do humans rep- 
resent “possible worlds” in their minds and compute 
the closest one, when the number of possibilities is far 
beyond the capacity of the human brain? Any credible 
theory of rationality must account for the astonishing 
ease with which humans comprehend, derive, and com- 
municate counterfactual information. Finally, it results 
in practical algorithms for solving some of the most crit- 
ical and difficult causal problems that have challenged 
data analysts and experimental researchers in the past 
century (see Pearl & Mackenzie, 2018, for an extensive 
historical account). I call this theory the structural causal 
model (SCM). 

In the rest of the chapter, we will focus on the proper- 
ties of SCM and explicate how it can be used to define 
counterfactuals (section 2), to control confounding and 
predict the effect of interventions and policies (sec- 
tion 3), to define and estimate direct and indirect effects 
(section 4), and, finally, to ensure generalizability of 
empirical results across diverse environments (section 5). 


2. Counterfactuals and SCM 


At the center of the structural theory of causation lies 
a “structural model,” M, consisting of two sets of vari- 
ables, Uand V, and a set F of functions that determine or 
simulate how values are assigned to each variable V;EV. 
Thus, for example, the equation 


Vi = filv, U) 


describes a physical process by which variable V; is assigned 
the value v; = fi(v,u) in response to the current values, 
v and uy, of all variables in V and U. Formally, the triplet 
(U, V,F) defines an SCM, and the diagram that captures 
the relationships among the variables is called the causal 
graph G (of M). The variables in U are considered “exog- 


enous,” namely, background conditions for which no 


Causal and Counterfactual Inference 


explanatory mechanism is encoded in model M. Every 
instantiation U = u of the exogenous variables uniquely 
determines the values of all variables in V, and hence, if 
we assign a probability P(u) to U, it defines a probability 
function P(v) on V. The vector U = u can also be inter- 
preted as an experimental “unit,” which can stand for an 
individual subject, agricultural lot, or time of day, since 
it describes all factors needed to make V a deterministic 
function of U. 

The basic counterfactual entity in structural models 
is the sentence “Y would be y had X been x in unit (or 
situation) U = u,” denoted Y,(u) = y. Letting M, stand for 
a modified version of M, with the equation(s) of set X 
replaced by X = x, the formal definition of the counter- 
factual Y,(u) reads 


¥,(u) = Yu,(u). (3) 


In words, the counterfactual Y,(u) in model M is defined 
as the solution for Y in the “modified” submodel M,. 
Galles and Pearl (1998) and Halpern (1998) have given 
a complete axiomatization of structural counterfactuals, 
embracing both recursive and nonrecursive models (see 
also Pearl, 2009b, chapter 7).* 

Since the distribution P(u) induces a well-defined 
probability on the counterfactual event Y, = y, it also 
defines a joint distribution on all Boolean combina- 
tions of such events, for instance, “Y.=y & Zy =z,” 
which may appear contradictory, if x « x’. For example, 
to answer retrospective questions, such as whether Y 
would be y, if X were x,, given that in fact Y is yo and 
X is Xo, we need to compute the conditional probability 
P(Y,,="%|Y=Vo, X= Xo), which is well defined once we 
know the forms of the structural equations and the dis- 
tribution of the exogenous variables in the model. 

In general, the probability of the counterfactual sen- 
tence P(Y, = y|e), where e is any propositional evidence, 
can be computed by the following three-step process 
(Pearl, 2009b, p. 207): 
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Step 1 (abduction): Update the probability P(u) to obtain 
P(ule). 

Step 2 (action): Replace the equations determining the 
variables in set X by X = x. 

Step 3 (prediction): Use the modified model to compute 
the probability of Y = y. 


In temporal metaphors, step 1 explains the past (U) in 
light of the current evidence e, step 2 bends the course 
of history (minimally) to comply with the hypothetical 
antecedent X = x, and finally, step 3 predicts the future 
(Y) based on our new understanding of the past and our 
newly established condition, X = x. 


2.1 Example: Computing Counterfactuals 

in Linear SCM 

We illustrate the working of this three-step algorithm 
using a linear structural equation model, depicted by the 
graph in figure 7.1.1. 

To motivate the analysis, let X stand for the level of 
assistance (or “treatment”) given to a student, Z stands 
for the amount of time the student spends studying, and 
Y, the outcome, stands for the student’s performance on 
the exam. The algebraic version of this model takes the 
form of the following equations: 


X= &, 
Z = PX+ €2, 
VY = OX+YZ+ 3. 


The coefficients a, 6, and y are called “structural coeffi- 
cients,” to be distinguished from regression coefficients, 
and represent direct causal effects of the correspond- 
ing variables. Under appropriate assumptions, say that 
the error terms ¢€), 2, and ¢; are mutually independent, 
the structural coefficients can be estimated from data. 
Our task, however, is not to estimate causal effects but 
to answer counterfactual questions taking the model as 
given. 


Figure 7.1.1 


Structural models used for answering a counterfactual question about an individual u = (€1, &, €3). (a) The generic model. (b) The 


u-specific model. (c) The modified model necessary to accommodate the antecedent Z = 2 of the counterfactual question Q). 
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Let us consider a student named Joe, for whom we 
measure X = 0.5, Z = 1, Y = 1.5, and about whom we ask 
a counterfactual question: 


Q,: What would Joe’s score have been had he doubled 
his study time? 


Using our subscript notation, this question amounts to 
evaluating Yz-2(u), with u standing for the distinctive 
characteristics of Joe, namely, u = (é1, &, €3), aS inferred 
from the observed data {X = 0.5, Z = 1, Y = 1.5}. 

Following the algorithm above, the answer to this 
question is obtained in three steps: 


1. Use the data to compute the exogenous factors 
£1, €2, €3. (These are the invariant characteristics of 
unit uv and do not change by interventions or coun- 
terfactual hypothesizing.) In our model, we get (see 
figure 7.1.1b): 


Ey =0.5 
é& = 1-0.5x0.5 = 0.75, 
€3 = 1.5-0.5x0.7-1x0.4 = 0.75. 


2. Modify the model, to form Mz- 2, in which Z is set to 
2 and all arrows to Z are removed (figure 7.1.1c). 

3. Compute the value of Y in the mutilated model 
formed in step 2, which gives 


Yz-2 =0.5x0.7+2.0x0.4+0.75 = 1.90. 


We can thus conclude that, had he doubled his study 
time, Joe’s score would have been 1.90 instead of 1.5. 
This example illustrates the need to modify the original 
model (figure 7.1.1a), in which the combination X = 1, 
&, = 0.75, Z = 2.0 constitutes a contradiction. 


2.2 The Two Principles of Causal Inference 

Before describing specific applications of the structural 
theory, it will be useful to summarize its implications 
in the form of two “principles,” from which all other 
results follow: 


Principle 1: The law of structural counterfactuals 


Principle 2: The law of structural independence 


The first principle is described in equation (3) and instructs 
us how to compute counterfactuals and their probabilities 
from a structural model. This, together with principle 2, 
will allow us (section 3) to determine what assumptions 
one must make about reality in order to infer probabili- 
ties of counterfactuals from either experimental or passive 
observations. 

Principle 2 defines how structural features of the model 
entail dependencies in the data. Remarkably, regardless 
of the functional form of the equations in the model and 
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regardless of the distribution of the exogenous variables 
U, if the latter are mutually independent and the model 
is recursive, the distribution P(v) of the endogenous vari- 
ables must obey certain conditional independence rela- 
tions, stated roughly as follows: whenever sets X and Y 
are “separated” by a set Z in the graph, X is indepen- 
dent of Y given Z (Verma & Pearl, 1988). This “separa- 
tion” condition, called “d-separation” (Geiger, Verma, & 
Pearl, 1990; Pearl, 2000a, pp. 16-18), constitutes the link 
between the causal assumptions encoded in the causal 
graph (in the form of missing arrows) and the observed 
data. It is defined formally as follows: 


Definition 1 (d-separation). A set S of nodes is said to 
block a path p if either 


1. pcontains at least one arrow-emitting node that 
is in S, or 

2. p contains at least one collider (i.e., a node 
obtaining head-to-head arrows) that is outside 
S and has no descendant in S. 


If S blocks all paths from set X to set Y, it is said to 
“d-separate X and Y,” and then variables X and Y are 
independent given S, written X 1 Y|S.* 


D-separation implies conditional independencies for 
every distribution P(v) that is compatible with the causal 
assumptions embedded in the diagram. To illustrate, 
the diagram in figure 7.1.2a implies Z; ll Y| (X,Z3, W2), 
because the conditioning set S = {X, Z3, W2} blocks all 
paths between Z; and Y. The set S = {X, Z3, W3}, how- 
ever, leaves the path (Z1,Z3,Z2,W2,Y) unblocked (by 
virtue of the collider at Z3), and so, the independence 
Z, I Y | (X,Z3, Ws) is not implied by the diagram. 


3. Intervention, Identification, and Causal Calculus 


To maximize the expectation defined by equation (1), 
a central problem for any rational agent is that of infer- 
ring the probability P,(y) from empirical data. In the 
context of social or medical policy making, this amounts 


Figure 7.1.2 
Illustrating the intervention do(X = x). (a) The original model M. 
(b) The intervention submodel M, and the counterfactual Y,. 
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to estimating the interventional probability P(y | do(x)), 
which is defined, using the counterfactual Y,, as° 


P(y | do(x)) P(x = y). (4) 


Given a model M, the effect of an intervention X = do(x) 
can be predicted from the submodel M, as shown in fig- 
ure 7.1.2. Figure 7.1.2b illustrates the submodel M, created 
by the atomic intervention do(x); it sets the value of X to 
x and thus removes the influence of W; and Z3 on X. We 
similarly define the result of conditional interventions by 


P(y | do(x), z) & PCy,z | do(x))/P(z | do(x)) 
= P(Y, = y|Zx = 2). (5) 


P(y | do(x), z) captures the z-specific effect of X on Y, that 
is, the effect of setting X to x among those units only for 
which Z = z. 

A second important question concerns identification 
in partially specified models: given a set A of qualita- 
tive causal assumptions, as embodied in the structure of 
the causal graph, can the controlled (postintervention) 
distribution, P(y|do(x)), be estimated from the available 
data that are governed by the preintervention distribu- 
tion P(z, x,y)? In linear parametric settings, the question 
of identification reduces to asking whether some model 
parameter, 8, has a unique solution in terms of the par- 
ameters of P (say the population covariance matrix). In the 
nonparametric formulation, the notion of “has a unique 
solution” does not directly apply since quantities such 
as Q = P(y|do(x)) have no parametric signature and are 
defined procedurally by a symbolic operation on the causal 
model M, as in figure 7.1.2b. The following definition cap- 
tures the requirement that Q be estimable from the data: 


Definition 2 (Identifiability; Pearl, 2000a, p. 77). A 
causal query Q is identifiable from data compatible 
with a causal graph G, if for any two (fully specified) 
models M,; and Mz that satisfy the assumptions in G, 
we have 


P,(v) = Po(v) = Q(M;) = Q(M2). (6) 


In words, equality in the probabilities P;(v) and P2(v) 
induced by models M, and Mz, respectively, entails equal- 
ity in the answers that these two models give to query 
Q. When this happens, Q depends on P only and should 
therefore be expressible in terms of the parameters of P. 
When a query Q is given in the form of a do-expression, 
for example, Q = P(y|do(x),z), its identifiability can be 
decided systematically using an algebraic procedure 
known as the “do-calculus” (Pearl, 1995). It consists of 
three inference rules that permit us to equate interven- 
tional and observational distributions whenever certain 
d-separation conditions hold in the causal diagram G. 
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3.1 The Rules of the Do-Calculus 
Let X, Y, Z, and W be arbitrary disjoint sets of nodes in a 
causal directed acyclical graph (DAG) G. We denote by Gz 
the graph obtained by deleting from G all arrows point- 
ing to nodes in X. Likewise, we denote by Gy the graph 
obtained by deleting from G all arrows emerging from 
nodes in X. To represent the deletion of both X-incoming 
and Z-outgoing arrows, we use the notation Gy,. 

The following three rules are valid for every interven- 
tional distribution compatible with G: 


Rule 1 (Insertion/deletion of observations): 


Ply | do(x), Z, w) = P(y | do(x), w) 
if VLZ|X,W)a,. (7) 


Rule 2 (Action/observation exchange): 


P(y| do(x), do(z), w) = P(y | do(x), z, w) 
if VUZ|X,W)g,,. (8) 


Rule 3 (Insertion/deletion of actions): 


P(y|do(x), do(z), w) = P(y| do(x), w) 3 
if YUZ|X,W)a, (9) 


Zw)’ 


where Z(W) is the set of Z-nodes that are not ances- 
tors of any W-node in G;. 


To establish identifiability of a causal query Q, one needs 
to repeatedly apply the rules of the do-calculus to Q, until 
an expression is obtained that no longer contains a do- 
operator;® this renders it estimable from nonexperimental 
data. The do-calculus was proven to be complete for que- 
ries in the form Q = P(y| do(x), z) (Huang & Valtorta, 2006; 
Shpitser & Pearl, 2006), which means that if Q cannot be 
reduced to probabilities of observables by repeated applica- 
tion of these three rules, then such a reduction does not 
exist, that is, the query is not estimable from observational 
studies without strengthening the assumptions. 


3.2 Covariate Selection: The Backdoor Criterion 

One of the most powerful results emerging from the 
do-calculus is a method of identifying a set of variables 
that, if measured, would permit us to predict the effect 
of action from passive observation. This set of variables 
coincides with the set Z of equation (2), which we called 
“past” in section 1 and will now receive a formal charac- 
terization in definition 3. 

Consider an observational study in which we wish to 
find the effect of some treatment (X) on a certain outcome 
(Y), and assume that the factors deemed relevant to the 
problem are structured as in figure 7.1.2a; some are affect- 
ing the outcome, some are affecting the treatment, and 
some are affecting both treatment and response. Some of 
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these factors may be unmeasurable, such as genetic trait 
or lifestyle, while others are measurable, such as gender, 
age, and salary level. Our problem is to select a subset of 
these factors for measurement and adjustment such that 
if we compare treated versus untreated subjects having 
the same values of the selected factors, we get the correct 
treatment effect in that subpopulation of subjects. Such a 
set of factors is called a “sufficient set” or a set “appropri- 
ate for adjustment” (see Greenland, Pearl, & Robins, 1999; 
Pearl, 2000b, 2009a). The following criterion, named 
“backdoor” (Pearl, 1993), provides a graphical method of 
selecting such a set of factors for adjustment: 


Definition 3 (Admissible sets—the backdoor crite- 
rion). A set S is admissible (or “sufficient”) for estimat- 
ing the causal effect of X on Y if two conditions hold: 


1. No element of S is a descendant of X. 


2. The elements of S “block” all “backdoor” paths 
from X to Y—namely, all paths that end with 
an arrow pointing to X. 


Based on this criterion, we see, for example, that in 
figure 7.1.2, the sets {Z1,Z2,Z3}, {Z1,Z3}, {Wi,Z3}, and 
{W2,Z3} are each sufficient for adjustment, because each 
blocks all backdoor paths between X and Y. The set {Z3}, 
however, is not sufficient for adjustment because it does 
not block the path X<— W, <= 2, > Z3< Z, >W2-Y. 

The intuition behind the backdoor criterion is as fol- 
lows. The backdoor paths in the diagram carry spuri- 
ous associations from X to Y, while the paths directed 
along the arrows from X to Y carry causative associations. 
Blocking the former paths (by conditioning on S) ensures 
that the measured association between X and Y is purely 
causal, namely, it correctly represents the target quantity: 
the causal effect of X on Y. Conditions for relaxing restric- 
tion 1 are given in Pearl (2009b, p. 338), Pearl and Paz 
(2014), and Shpitser, VanderWeele, and Robins (2010).’ 

The implication of finding a sufficient set, S, is that 
stratifying on S is guaranteed to remove all confound- 
ing bias relative to the causal effect of X on Y. In other 
words, it renders the causal effect of X on Y identifiable, 
via the adjustment formula® 


P(Y=y|do(X=x)) 


=) P(¥=y|X=x, S=s)P(S=5). ) 


Since all factors on the right-hand side of the equation 
are estimable (e.g., by regression) from preinterventional 
data, the causal effect can likewise be estimated from 
such data without bias. Note that equation (2) is a special 
case of equation (10), where S is chosen to include all 
variables preceding X in the causal order. Moreover, the 
backdoor criterion implies the independence X 1 Y,|5S, 
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also known as “conditional ignorability” (Rosenbaum & 
Rubin, 1983), and provides therefore the scientific basis 
for most inferences in the potential-response framework. 

The backdoor criterion allows us to write equation (10) 
by inspection, after selecting a sufficient set, S, from the 
diagram. The selection criterion can be applied system- 
atically to diagrams of any size and shape, thus freeing 
analysts from judging whether “X is conditionally ignor- 
able given S,” a formidable mental task required in the 
potential-response framework. The criterion also enables 
the analyst to search for an optimal set of covariates— 
namely, a set S that minimizes measurement cost or sam- 
pling variability (Tian, Paz, & Pearl, 1998). 


Theorem 1 (identification of interventional expres- 
sions). Given a causal graph G containing both 
measured and unmeasured variables, the consistent 
estimability of any expression of the form 


Q = PUV1, V2,» + Vin | dO(X1, Xz).  Xn)y Zp Za). - +» Zk) 


can be decided in polynomial time. If Q is estimable, 
then its estimand can be derived in polynomial time. 
Furthermore, the algorithm is complete. 


The results stated in theorem 1 were developed in sev- 
eral stages over the past 20 years (Pearl, 1993, 1995; Shpit- 
ser & Pearl, 2006; Tian & Pearl, 2002). Bareinboim and 
Pearl (2012a) extended the identifiability of Q to combi- 
nations of observational and experimental studies. 

It is important to note at this point that the do- 
operator can be used not merely for fixing a variable 
at a predetermined value x but also for analyzing “soft 
interventions.” For example, the effect of additive inter- 
ventions, such as “administer 5 mg of insulin to a given 
patient,” can be estimated using the do-calculus (Pearl, 
Glymour, & Jewell, 2016, p. 109). Likewise, the effects 
of stochastic interventions (e.g., “change the frequency 
with which this patient receives a drug”) can be esti- 
mated by a method based on the do-operator (Pearl, 
2009b, p. 113). The versatility of the do-operator is fur- 
ther discussed in Pearl (2009b, section 11.4). 


4. Mediation Analysis 


Mediation analysis aims to uncover causal pathways 
along which changes are transmitted from causes to 
effects. Interest in mediation analysis stems from both 
scientific and practical considerations. Scientifically, 
mediation tells us “how nature works,” and practically it 
enables us to predict behavior under a rich variety of con- 
ditions and policy interventions. For example, in cop- 
ing with the age-old problem of gender discrimination 
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(Bickel, Hammel, & O’Connell, 1975; Goldberger, 1984), 
a policy maker may be interested in assessing the extent 
to which gender disparity in hiring can be reduced by 
making hiring decisions gender-blind, compared with 
eliminating gender inequality in education or job quali- 
fications. The former concerns the “direct effect” of 
gender on hiring while the latter concerns the “indirect 
effect” or the effect mediated via job qualification. 

The role that mediation analysis plays in ratio- 
nal decision making revolves around the richer set of 
options that emerges from understanding “how the 
world works.” For example, the option of using mos- 
quito nets was not considered by decision makers when 
malaria was believed to be caused by malaria (“bad 
air”). It was fairly rational in those days to use breath- 
ing masks in swampy areas. It is hardly rational today, 
given the overwhelming evidence about the mediating 
effect of the Anopheles mosquito. The logic of properly 
accounting for empirical data in one’s belief system is 
an aspect of rational behavior that is gaining increased 
attention among researchers (Pearl, 2013). 

The structural model for a typical mediation problem 
takes the form 


t= frlur), 


where T (treatment), M (mediator), and Y (outcome) are 
discrete or continuous random variables; fr, fu, and fy 
are arbitrary functions; and Ur, Uy, and Uy represent, 
respectively, omitted factors that influence T, M, and Y. 
In figure 7.1.3a, the omitted factors are assumed to be 
arbitrarily distributed but mutually independent. In fig- 
ure 7.1.3b, the dashed arcs connecting Ur and Uy (as well 
as Uy and Ur) encode the understanding that the factors 
in question may be dependent. 


m= fault, um), y= fr(t, m, uy), (11) 


4.1 Natural Direct and Indirect Effects 

Using the structural model of equation (11), four types 
of effects can be defined for the transition from T = 0 to 
T=12 


Figure 7.1.3 
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Total effect: 

TE = E[fvl1, fu(1, uw), uy] — frl0, flO, ud), uy] 
=E[Y,-Yol (12) 


= E[Y| do(T = 1)] - E[Y| do(T = 0)]. 


TE measures the expected increase in Y as the treatment 
changes from T = 0 to T = 1, while the mediator is allowed 
to track the change in T as dictated by the function fi. 


Controlled direct effect: 


CDE(m) = E{fy[1,M = m, uy] - flO, M = m, uy} 
= E[Yim = Yoml (13) 
= E[Y|do(T = 1,M =m)] - E[Y|do(T = 0,M =m). 


CDE measures the expected increase in Y as the treat- 
ment changes from T = 0 to T = 1, while the mediator is 
set to a specified level M = m uniformly over the entire 
population. 


Natural direct effect:'° 


NDE = E{f, (1, fu0, Uy), ur] ~ fr[0, fu(0, Uy), u,|} (14) 
= ELY, Mo — Yo, mol - 


NDE measures the expected increase in Y as the treat- 
ment changes from T = 0 to T = 1, while the mediator 
is set to whatever value it would have attained (for each 
individual) prior to the change, that is, under T = 0. 


Natural indirect effect: 


NIE = E{f,[0, fu, Un), Uy] — fr [0, fu(O, Uy), uy} (15) 
= ELYo ua, Yo Mol - 


NIE measures the expected increase in Y when the treat- 
ment is held constant, at T = 0, and M changes to what- 
ever value it would have attained (for each individual) 
under T = 1. It captures, therefore, the portion of the 
effect that can be explained by mediation alone while 
disabling the capacity of Y to respond to X. 

We note that, in general, the total effect can be 
decomposed as 


(b) 


(a) The basic nonparametric mediation model, with no confounding. (b) A confounded mediation model in which dependence 


exists between U,, and (U;, Uy). 
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TE = NDE - NIE, (16) 


where NIE, stands for the natural indirect effect under 
the reverse transition, from T = 1 to T = 0. This implies 
that NIE is identifiable whenever NDE and TE are iden- 
tifiable. In linear systems, where reversal of transitions 
amounts to negating the signs of their effects, we have 
the standard additive formula, TE = NDE+ NIE. 

We further note that TE and CDE(m) are do-expressions 
and can, therefore, be estimated from experimental data. 
Not so NDE and NIE; both are counterfactual expressions 
that cannot be reduced to do-expressions. The reason is 
simple: there is no way to disable the direct effect by 
intervening on any variable in the model. The counter- 
factual language permits us to circumvent this difficulty 
by (figuratively) changing T to affect M while feeding Y 
the prior value of T. 

Since theorem 1 assures us that the identifiability of any 
do-expression can be determined by an effective algorithm, 
TE and CDE(m) can be identified by those algorithms. NDE 
and NIE, however, require special analysis, given in the 
next subsection. 


4.2 Sufficient Conditions for Identifying 
Natural Effects 
The following is a set of assumptions or conditions, 
marked A-1 to A-4, that are sufficient for identifying 
both direct and indirect natural effects. Each condition 
is communicated using the causal diagram. 

There exists a set W of measured covariates such that 


A-1 No member of W is a descendant of T. 

A-2 W blocks all backdoor paths from M to Y (not tra- 
versing X >MandX—Y). 

A-3 The W-specific effect of T on M is identifiable (using 
theorem 1 and possibly using experiments or auxil- 
iary variables). 

A-4 The W-specific joint effect of {T, M} on Y is identi- 
fiable (using theorem 1 and possibly using experi- 
ments or auxiliary variables). 


Theorem 2 (identification of natural effects). When 
conditions A-1 and A-2 hold, the natural direct effect 
is experimentally identifiable and is given by 


NDE = 9) °[E(Y | do(T =1, M=m), W=w) 
"" _E(Y|do(T=0,M=m),W=w)| (17) 
x P(M =m |do(T=0),W=w)x P(W=w). 


The identifiability of the do-expressions in equation 
(17) is guaranteed by conditions A-3 and A-4 and can be 
determined by theorem 1. 
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In the nonconfounding case (figure 7.1.3a), NDE 
reduces to 

NDE => [E(Y |T=1, M=m)- E(Y|T=0, M=m)| 

m (18) 
x P(M=m|T=0), 
which came to be known as the mediation formula (Pearl, 
2012). 

Shpitser (2013) further provides complete algorithms 
for identifying natural direct and indirect effects and 
extends these results to path-specific effects with mul- 
tiple treatments and multiple outcomes. 


5. External Validity and Transportability 


To support the choice of optimal actions on the basis of 
nonexperimental data, the role of the do-calculus is to 
remove the do-operator from the query expression. We 
now discuss a totally different application, to decide if 
experimental findings from environment z can be trans- 
ported to a new, potentially different environment a*, 
in which only passive observations can be performed. 
This problem, labeled “transportability” in Pearl and 
Bareinboim (2011), is at the heart of every scientific 
investigation since, invariably, experiments performed 
in one environment (or population) are intended to be 
used elsewhere, where conditions may differ. 

To formalize problems of this sort, a graphical repre- 
sentation called [a] “selection diagram” was devised (fig- 
ure 7.1.4), which encodes knowledge about differences 
and commonalities between populations. A selection dia- 
gram is a causal diagram annotated with new variables, 
called “S-nodes,” which point to the mechanisms where 
discrepancies between the two populations are suspected 
to be located. The task of deciding if transportability is fea- 
sible now reduces to the syntactic problem of separating 
(using the do-calculus) the do-operator from the S-variables 
in the query expression P(y| do(x), z, s). In effect, this sepa- 
ration renders the disparities irrelevant to what we learn 
in the experimental setup. 


Theorem 3 (Pearl & Bareinboim, 2011). Let D be the 
selection diagram characterizing two populations, z 
and z*, and S a set of selection variables in D. The rela- 
tion R = P*(y|do(x), z) is transportable from wand a* if 
and only if the expression P(y | do(x), z, s) is reducible, 
using the rules of the do-calculus, to an expression in 
which S appears only as a conditioning variable in do- 
free terms. 


While theorem 3 does not specify the sequence of 
rules leading to the needed reduction (if such exists), a 
complete and effective graphical procedure was devised 
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(a) (b) 


Figure 7.1.4 


Selection diagrams depicting differences in populations. (a) The two populations differ in age distributions. (b) The populations 


differ in how reading skills (Z) depend on age (an unmeasured variable, represented by the hollow circle), and the age distributions 


are the same. (c) The populations differ in how Z depends on X. Dashed arcs (e.g., X<-—-—> Y) represent the presence of latent 


variables affecting both X and Y. 


by Bareinboim and Pearl (2012b), which also synthesizes 
a transport formula whenever possible. Each transport for- 
mula determines what informations need to be extracted 
from the experimental and observational studies and how 
they ought to be combined to yield an unbiased estimate 
of the relation R = P(y| do(x),s) in the target population z*. 
For example, the transport formulas induced by the three 
models in figure 7.1.4 are given by 


P(y |do(x),s) = ¥ P(y| do(x), z)P(z|s), (a) 
P(y|do(x),s) = P(y | do(x)), (b) 
P(y|do(x),s) = Y Ply |do(x),z)P(z |X,S). (c) 


Each of these formulas satisfies theorem 3, and each 
describes a different procedure of pooling information 
from zand x*. 

For example, (c) states that to estimate the causal effect 
of X on Y in the target population a*, P(y|do(x),z,s), we 
must estimate the z-specific effect P(y|do(x),z) in the 
source population a and average it over z, weighted by 
P(z|x,s), that is, the conditional probability P(z| x) esti- 
mated at the target population z*. The derivation of this 
formula follows by writing 


P(y | do(x), s) = Py | do(x), z, s)P(z | do(x), s) 


and noting that rule 1 of the do-calculus authorizes the 
removal of s from the first term (since Y 1 S|Z holds 
in Gy), and rule 2 authorizes the replacement of do(x) 
with x in the second term (since the independence Z 1 X 
holds in Gy). 

A generalization of transportability theory to mul- 
tiple environments has led to a method called “data 
fusion” (Bareinboim & Pearl, 2016), aimed at combin- 
ing results from many experimental and observational 
studies, each conducted on a different population and 
under a different set of conditions, so as to synthesize an 


aggregate measure of effect size in yet another environ- 
ment, different from the rest. This fusion problem has 
received enormous attention in the health and social 
sciences, where it is typically handled inadequately by 
a statistical method called “meta-analysis,” which “aver- 
ages out” differences instead of rectifying them. 

Using multiple “selection diagrams” to encode com- 
monalities among studies, Bareinboim and Pearl (2013) 
“synthesized” an estimator that is guaranteed to provide 
an unbiased estimate of the desired quantity based on 
information that each study shares with the target envi- 
ronment. Remarkably, a consistent estimator may be 
constructed from multiple sources even in cases where 
it is not constructible from any one source in isolation. 


Theorem 4 (Bareinboim & Pearl, 2013). 


e Nonparametric transportability of experimental 
findings from multiple environments can be deter- 
mined in polynomial time, provided suspected dif- 
ferences are encoded in selection diagrams. 

e When transportability is feasible, a transport for- 
mula can be derived in polynomial time that speci- 
fies what information needs to be extracted from 
each environment to synthesize a consistent esti- 
mate for the target environment. 


e The algorithm is complete, that is, when it fails, 
transportability is infeasible. 


Another problem that falls under the data fusion 
umbrella is that of “selection bias” (Bareinboim, Tian, 
& Pearl, 2014), which requires a generalization from a 
subpopulation selected for a study to the population at 
large, the target of the intended policy. 

Selection bias is induced by preferential selection of 
units for observation, usually governed by unknown 
factors, thus rendering the data no longer representative 
of the environment (or population) of interest. Selection 
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bias represents a major obstacle to valid causal and sta- 
tistical inferences. It cannot be removed by randomized 
experiments and can rarely be detected in either experi- 
mental or observational studies.'' For instance, in a typi- 
cal study of the effect of training programs on earnings, 
subjects achieving higher incomes tend to report their 
earnings more frequently than those who earn less. The 
data-gathering process in this case will reflect this distor- 
tion in the sample proportions, and since the sample is 
no longer a faithful representation of the population, 
biased estimates will be produced regardless of how many 
samples were collected. Our ability to eliminate such bias 
by analytical means thus provides a major opportunity to 
the empirical sciences. 


6. Conclusions 


Rational decisions demand rational assessments of the 
likely consequences of one’s actions. This chapter offers 
a formal and normative account of how such assess- 
ments should be shaped by empirical observations and 
by prior world knowledge. The account is based on mod- 
ern research in causal inference, which extends beyond 
probability and statistics and is becoming, in my opin- 
ion, an integral part of the theory of rationality. 

One of the crowning achievements of modern work on 
causality has been to formalize counterfactual reasoning 
within a structure-based representation, the very repre- 
sentation researchers use to encode scientific knowledge. 
We showed that every structural equation model deter- 
mines the truth value of every counterfactual sentence. 
Therefore, we can determine analytically if the probabil- 
ity of a counterfactual sentence is estimable from experi- 
mental or observational studies or a combination thereof. 

This enables us to infer the behavior of specific indi- 
viduals, identified by a distinct set of characteristics, as 
well as the average behavior of populations, identified by 
preintervention features or postintervention response. 
Additionally, this formalization leads to a calculus of 
actions that resolves some of the most daunting problems 
in the empirical sciences. These include, among others, 
the control of confounding, the evaluation of interven- 
tional policies, the assessment of direct and indirect 
effects, and the generalization of empirical results across 
heterogeneous environments. The same calculus can be 
leveraged to generate rational explanations for action rec- 
ommended or actions taken in the past. 
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Notes 


1. The relation between P, and P takes a variety of equivalent 
forms, including the backdoor formula, truncated factoriza- 
tion, adjustment for direct causes, or the inverse probability 
weighting shown in equation (2) (Pearl, 2000a, pp. 72-73). 
The latter form is the easiest to describe without appealing to 
graphical notation. But see equation (10) in section 3.1 for a 
more general formula and definition 3 for a formal definition 
of the set Z. 


2. I will use “future” and “past” figuratively; “affected” and 
“unaffected” (by X) are more accurate technically (i.e., descen- 
dants and nondescendants of X, in graph-theoretical terminol- 
ogy). The derivation of equation (2) requires that processes be 
organized recursively (avoiding feedback loops); more intricate 
formulas apply to nonrecursive models. See Pearl (2009b, 
pp. 72-73) or Spirtes, Glymour, & Scheines (2001) for a simple 
derivation of this and equivalent formulas. Equation (2) has also 
been anticipated in Spohn (1978, sections 3.3 and 5.2). 


3. The structural definition of counterfactuals given in equa- 
tion (3) was first introduced in Balke and Pearl (1995). 


4. By a “path,” we mean a sequence of consecutive edges in 
the graph regardless of direction. See Pearl (2009b, p. 335) for 
a gentle introduction to d-separation and its proof. In linear 
models, the independencies implied by d-separation are valid 
for nonrecursive models as well. 


5. An alternative definition of do(x), invoking population aver- 
ages only, is given in Pearl (2009b, p. 24). 


6. Such derivations are illustrated in graphical details in Pearl 
(2009b, p. 87). 


7. In particular, the criterion devised by Pearl and Paz (2014) 
simply adds to condition 2 of definition 3 the requirement that 
X and its nondescendants (in Z) separate its descendants (in Z) 
from Y. 


8. Summations should be replaced by integration when applied 
to continuous variables, as in Imai, Keele, and Yamamoto 
(2010). 


9. Generalizations to arbitrary reference points, say from T = t to 
T =t', are straightforward. These definitions apply at the popu- 
lation levels; the unit-level effects are given by the expressions 
under the expectation. All expectations are taken over the factors 
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Uy and Uy. Note that in this section, we use parenthetical nota- 
tion for counterfactuals, replacing the subscript notation used in 
sections 2 and 3. 


10. Natural direct and indirect effects were conceptualized in 
Robins and Greenland (1992) and were formalized using equa- 
tions (14) and (15) in Pearl (2001). 


11. Remarkably, selection bias can be detected by combining 
experimental and observational studies, if certain coherence 
inequalities are violated (Pearl, 2009b, p. 294). 
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7.2 The Rationality of Everyday Causal Cognition 


Michael R. Waldmann 


Summary 


Normative and descriptive theories of causal cognition 
are tightly connected. Theories of causal cognition pro- 
posed in psychology have in most cases precursors in 
philosophy and other normative disciplines. For exam- 
ple, associative theories, causal Bayes net theories, and 
dispositional theories of causal cognition all are inspired 
by positions developed in philosophy. The competi- 
tion between normative accounts is therefore mirrored 
in psychology. Findings that seem rational within one 
normative account may appear deficient within a com- 
peting theory. Consistency with a specific normative 
account is, however, only one criterion to assess rational- 
ity; to assess rationality, also correspondence of causal 
representations to causal relations in the world has 
been used. The present chapter will selectively review 
research, focusing on causal reasoning and learning, to 
demonstrate the tight coupling between normative and 
descriptive theories of everyday causal cognition. 


1. Normative and Descriptive Theories 


Causal cognition belongs to our most central cognitive 
competencies. Without causal knowledge, we would 
not be able to make predictions and diagnoses, generate 
explanations, design artifacts, solve problems, or plan 
and act. The ubiquity of causal cognition has attracted 
scientists from various disciplines to this topic. Philos- 
ophers have studied causality for centuries, but more 
recently, the topic has also motivated research in the 
fields of psychology, artificial intelligence, anthropol- 
ogy, biology, economics, physics, and statistics (among 
others). Causality is a genuinely interdisciplinary topic 
attracting both researchers interested in developing nor- 
mative methods of causal analysis and researchers pursu- 
ing the descriptive goal of understanding how humans 
(in different age groups) and nonhuman animals reason 
about causal relations (for overviews, see Waldmann, 
2017; Waldmann & Hagmayer, 2013). 


Yet until recently, causal reasoning has been curi- 
ously absent from mainstream cognitive psychology. 
Although there has been some limited work in more spe- 
cific areas, such as developmental or social psychology, 
it did not play a significant role in fundamental theories 
of human cognition. To date, textbooks on cognitive 
psychology do not contain chapters on causal reason- 
ing. One reason for this neglect is that psychology was 
for many decades dominated by the view that cognitive 
mechanisms, such as associative learning, are general 
and can be specified independently from the domains 
to which they are applied. In many areas that addressed 
knowledge acquisition, domain-general similarity-based 
theories dominated, which neglected the role of causal 
knowledge (e.g., categorization, induction, conditional 
reasoning, learning). The situation has slowly changed 
in the past two decades, with more and more research 
devoted to the particular characteristics of reasoning 
and learning about causal systems. In fact, the relevance 
of causal knowledge has now been acknowledged in 
virtually all fields of higher-level cognition (see Wald- 
mann, 2017). The breadth of this literature makes it nec- 
essary to restrict the focus in the present chapter, which 
will primarily discuss general theoretical frameworks and 
empirical studies on causal reasoning and learning. 

Theories of causal cognition proposed in psychology 
have in most cases precursors in philosophy and other 
normative disciplines. It is not an accident that the goals 
of normative and descriptive theories of causality over- 
lap. Both scientists and laypeople intend their causal 
theories to be correct. Causal claims are therefore gener- 
ally associated with normative force (see Spohn, 2002; 
Waldmann, 2011). The common goals of normative and 
descriptive accounts of causal cognition may be the rea- 
son why psychologists often turn to normative theories 
as an inspiration for psychological theories. One of the 
most recent examples are causal Bayes nets, which were 
first developed in philosophy and engineering (see Pearl, 
1988, 2000; Spirtes, Glymour, & Scheines, 2001; Spohn, 
2012) but have also been adopted by psychologists as 
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models of everyday causal reasoning (for reviews, see 
Rottman, 2017; Rottman & Hastie, 2014; Waldmann, 
2017; Waldmann & Hagmayer, 2013; see also chapter 7.1 
by Pearl, this handbook). 

Although scientists and laypeople have common 
goals, it is also plausible to expect differences in their 
methodological approaches and the resulting types of 
representations of causal knowledge. Causal domains 
can fundamentally differ, so that a method that has been 
developed to acquire knowledge in economics or soci- 
ology, for example, differs from methods employed in 
chemistry or physics. Causal Bayes nets (see chapter 4.2 
by Hartmann, this handbook), which were intended as 
general models of causation, are more often used in the 
social sciences than in physics. Similarly, laypeople use 
causal knowledge in various everyday domains, includ- 
ing intuitive psychology, biology, and sociology. Again, 
although there are attempts to develop abstract theories 
of causality, such as Bayes nets, to model knowledge in 
these different areas, there are also more specific theories 
that focus on unique domain properties (e.g., intuitive 
physics or belief-desire psychology). Moreover, everyday 
knowledge is often rudimentary, erroneous, and driven 
by unsubstantiated beliefs (see Rozenblit & Keil, 2002; 
Sloman & Fernbach, 2017). Nevertheless, causal knowl- 
edge is probably adequate more often than not; other- 
wise, it would not be adaptive. 

Another difference between normative and descrip- 
tive approaches is that philosophers and scientists inter- 
ested in methodology generally try to develop a uniform 
coherent account based on few fundamental principles. 
By contrast, laypeople are often satisficers (Simon, 1956): 
they use tools that work for a given problem, but they 
are rarely interested in achieving coherence and consis- 
tency (see Arkes, Gigerenzer, & Hertwig, 2016; see also 
chapter 8.5 by Hertwig & Kozyreva, this handbook). 


2. How Rational Is Everyday Causal Cognition? 


Given that most psychological theories of causal cogni- 
tion have been inspired by normative accounts, most 
research addresses, at least implicitly, the question of 
whether everyday causal cognition conforms to the nor- 
mative principles developed in philosophy, statistics, or 
machine learning. However, the normative evaluation 
of causal cognition is complicated by the fact that there 
is no unique normative account of causation that is uni- 
versally accepted. The competition within the normative 
discourse is reflected in psychological theories. For exam- 
ple, associative theories, which can be traced back to the 
philosopher David Hume (1748/1977), model causation 
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as statistical covariation, whereas causal Bayes nets view 
associative approaches as deficient and argue that they 
neglect important specific features of causality. A further 
complexity is added by the fact that even within each 
of these frameworks, different competing normative and 
descriptive theories have been developed. Thus, depend- 
ing on the normative view of the researcher, findings 
can be seen either as confirming or as violating norms 
(see the following sections for examples). 

When researchers discover deviations from initially 
held normative accounts, different strategies are pur- 
sued. Sometimes the findings lead to modifications of 
the theory with the primary goal of maximizing descrip- 
tive adequacy (e.g., different versions of associative theo- 
ries). At other times, the strategy is to develop a modified 
normative theory by adding features that had previously 
been neglected (e.g., sensitivity to uncertainty in Bayes- 
ian theories). These attempts seek to be both descrip- 
tively adequate and normative. 

Whereas the debates about which norm is appropri- 
ate address rationality in terms of coherence with spe- 
cific norms, another perspective on rationality focuses 
on whether causal cognition corresponds to objective 
relations in the world (the correspondence view of ratio- 
nality; Arkes et al., 2016; see also chapter 8.5 by Her- 
twig & Kozyreva, this handbook). Thus, one important 
question is whether people tend to distort observations 
and whether possible distortions can be rationalized 
as adaptive. There is evidence showing that learners 
are often poor in correctly encoding probability infor- 
mation (Rottman & Hastie, 2014) and tend to over- or 
underestimate covariations, especially when these are 
zero (for a review, see Waldmann & Hagmayer, 2013). 
There have been attempts to argue that some of these 
distortions may actually be rational because they reflect 
the integration of prior knowledge, but not all cases can 
be explained this way. 


3. Frameworks of Causal Cognition 


The plurality of philosophical theories of causality is also 
reflected in psychological approaches. Various frame- 
works differ in how they model causality and causal cog- 
nition (see also Waldmann & Mayrhofer, 2016). I will use 
the term “framework” to describe classes of theories that 
use substantially different theoretical concepts to cap- 
ture causality. Frameworks also differ in the tasks they 
are trying to model. In this section, I will describe dif- 
ferent frameworks applied in causal reasoning research. 
The main distinguishing features of frameworks are the 
proposed causal relata (that is, the type of entities that 
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enter causal relations) and the way causal relations are 
modeled. 


3.1 The Dependency Framework 

The dependency view of causation is adopted by several 
psychological theories, which share central framework 
assumptions but otherwise often compete. Theories 
that can be subsumed under the dependency framework 
include associative theories (see Le Pelley, Griffiths, & Bees- 
ley, 2017), covariation theories (e.g., Cheng & Novick, 
1992; Perales, Catena, Candido, & Maldonado, 2017), 
power probabilistic contrast (PC) theory (Cheng, 1997), 
causal model theories (e.g., Gopnik et al., 2004; Rehder, 
2017a, 2017b; Rottman, 2017; Sloman, 2005; Wald- 
mann & Hagmayer, 2001; Waldmann & Holyoak, 1992; 
Waldmann, Holyoak, & Fratianne, 1995), and Bayesian 
inference theories (Griffiths & Tenenbaum, 2005, 2009; 
Lu, Yuille, Liljeholm, Cheng, & Holyoak, 2008; Meder, 
Mayrhofer, & Waldmann, 2014). 

According to dependency theories, a variable C is a 
cause of variable E if E statistically depends upon C. There 
is an extensive debate in philosophy about the proper 
causal relata in dependency theories (e.g., events, propo- 
sitions, facts, properties, or states of affairs; see Ehring, 
2009; Spohn, 2012). Causal relations are often graphically 
depicted by causal arrows directed from cause to effect 
(see figure 7.2.1). For example, a causal model could be 
postulated that uses the binary variables representing 
the effect forest fire (present vs. absent) and as potential 
causes a lit match (e.g., dropped by an arsonist vs. not 
dropped) and a lightning (present vs. absent) (see the 
common-effect model in figure 7.2.1a). The dependen- 
cies, then, encode a set of hypothetical situations con- 
sistent with the causal model. Causal models can also 
express mechanism knowledge as chains or networks of 
mediating variables. 


Representative research topics 

Associative theories One early popular class of mod- 
els of causal cognition are associative theories, which 
mainly focus on learning. Inspired by the philosopher 
David Hume (1748/1977), the key claim is that causal 
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Two examples of causal Bayes nets: (a) common-effect model 
and (b) common-cause model. 


learning is based on associating repeatedly observed spa- 
tiotemporally contiguous event pairs. How exactly such 
associations arise, and which covariation measures they 
approximate, is a matter of great debate between compet- 
ing associative theories (see Le Pelley et al., 2017; Perales 
et al., 2017). 

Whereas Hume additionally assumed that causes and 
effects that enter associations are ordered in the temporal 
order of their occurrence in the world (i.e., causes precede 
their effects), psychological theories of association focus 
on the temporal order of learning cues and outcomes, 
which can refer either to causes or to effects. For exam- 
ple, for a physician, a cue could be an observed symptom 
and the outcome the diagnosis of a disease. In this case, 
the cue refers to an effect of its cause, the disease. 


Associative theories versus causal model theory Whereas 
in associative theories, weights are acquired in the 
cue-outcome direction regardless of whether cues 
represent causes or effects, a crucial feature of causal 
models is that they contain arrows that represent 
causal directionality regardless of the order in which 
information about the variables is acquired (see fig- 
ure 7.2.1). This crucial difference led to a debate in 
the 1990s about whether people are sensitive to causal 
directionality and use causal model representations or 
whether they disregard this crucial feature of causality 
and acquire associative knowledge in the cue-outcome 
direction, analogous to multiple regression analysis in 
statistics. Thus, this debate was about which normative 
principle people apply in everyday causal cognition. 
Waldmann and Holyoak (1992) presented evidence 
supporting the causal model perspective (see also 
Waldmann, 1996, 2000; Waldmann et al., 1995). In 
the meantime, much more evidence for causal model 
representations has been collected (e.g., Gopnik et al., 
2004; Rehder, 2017a, 2017b; Sloman, 2005), although 
there is also some evidence showing that in complex 
learning tasks, learners sometimes fall back to simpler 
associative representations (see, e.g., L6pez, Cobos, & 
Cafio, 2005). 


Estimating causal strength Another debate related to a 
normative discussion about the proper norm concerns 
how causal strength parameters should be estimated. 
Most associative theories assume that causal strength is 
equal to the observed covariation. A popular measure 
for single cause-effect relations, used both by rule-based 
theories and by associative theories such as the Rescorla— 
Wagner rule (Rescorla & Wagner, 1972), is delta-p, which 
is the difference between the probability of an effect 
when its cause is present and the probability of the effect 
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when its cause is absent. Conceptually, this measure 
views causes as difference makers. 

Although delta-p is an accepted covariation measure, 
Cheng (1997) has questioned its normative and psy- 
chological suitability as a measure of causal strength. 
Associative theories add up multiple associative weights 
when multiple causes are present, which may lead to the 
nonsensical prediction that an effect will occur with a 
probability higher than 1. Thus, possible ceiling effects 
are not accounted for. Another example of a problem- 
atic ceiling effect are cases in which the probability of 
the effect in the absence of the cause is already at maxi- 
mum (i.e., 1) so that the cause cannot further increase 
the probability of the effect. Concluding from such 
an observation that the cause has zero strength seems 
counterintuitive. 

Cheng (1997) has therefore suggested a new measure 
of causal strength, causal power, which represents the 
probability with which a cause, acting alone, generates 
or prevents a target effect. Given that causes are never 
observed alone, causal power is a theoretical quantity 
that needs to be inferred on the basis of observed data. 
Cheng’s theory uses delta-p along with default back- 
ground assumptions about the underlying causal model 
(e.g., independence of target cause and unobserved alter- 
native causes) to infer the causal power parameter. 

Cheng’s (1997) theory is not the only theory of causal 
strength estimation that claims to be both normative 
and descriptively adequate. Cheng’s goal was to develop 
a theory that generates point estimates of causal power. 
By contrast, Griffiths and Tenenbaum (2005) used the 
framework of Bayesian inference theory and analyzed 
causal inference in the context of their causal support 
model, which incorporates uncertainty of parameter 
estimates by considering distributions of parameters. 
Whereas Cheng’s (1997) focus were judgments of causal 
strength, which were modeled as parameter estimation 
tasks, the causal support model focuses on the assess- 
ment of the likelihood of the presence of a causal link 
between a target cause C and an effect E. Griffiths and 
Tenenbaum argue that this is actually the question learn- 
ers try to answer when asked to assess causal strength. 
The causal support model aims to contrast the evidence 
in favor of a causal model in which a link exists between 
C and E with a causal model in which these two variables 
are independent and E is only influenced by background 
causes. One notable feature of the model is that it can 
rationally explain why people sometimes offer positive 
numerical estimates when they have observed contin- 
gencies that are actually at zero. 
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Observing versus intervening A unique feature that sets 
causal Bayes nets apart from associative theories is their 
ability to predict the outcomes of interventions based 
on observational knowledge alone (Pearl, 2000; Spirtes 
et al., 2001; see also chapter 7.1 by Pearl, this hand- 
book). Assume, for example, that we have observed that 
atmospheric pressure, barometer readings, and weather 
covary and that we have reason to believe that atmo- 
spheric pressure is the common cause of the two effects. 
Both causal Bayes nets and associative theories would 
allow us to make observational predictions between the 
three events. If we observe, for example, high barometer 
readings, we are licensed to predict good weather. How- 
ever, without further observations, causal Bayes nets 
also allow us to correctly predict that intervening on the 
barometer (for example, by shaking it) and thus chang- 
ing the reading would not alter the weather. According to 
Bayes nets accounts, we would represent such an action 
as the result of a do-operator (Pearl, 2000) that is the sole 
cause of the new reading, whereas the previous cause, 
atmospheric pressure, is now causally disconnected from 
the barometer. In the common-cause model, this would 
amount to removing the arrow between the atmospheric 
pressure variable and the barometer variable (“graph sur- 
gery”). Using this method, more complex planning situ- 
ations can also be modeled (see Pearl, 2000). Associative 
theories, by contrast, can only predict that intervening on 
the barometer leads to outcomes that are different from 
those in situations in which the barometer is simply 
observed, when the learner has actually intervened in 
the past and observed the associated outcome (i.e., inter- 
ventional learning); these theories cannot make correct 
interventional predictions based on observations alone. 

The distinction between observing and intervening 
represents an interesting test case to distinguish between 
competing theories within the dependency framework. 
Therefore, a number of psychologists have investigated 
whether human subjects are capable of deriving inter- 
ventional predictions from mere observations (Meder, 
Hagmayer, & Waldmann, 2008, 2009; Sloman & Lag- 
nado, 2005; Waldmann & Hagmayer, 2005). The general 
finding in these studies is that participants proved capa- 
ble of deriving interventional predictions from observa- 
tions, although in more complex tasks, some participants 
seemed to confuse observational with interventional 
knowledge. Moreover, there are also some preliminary 
findings suggesting that rats, too, are capable of deriving 
interventional predictions from observations (Blaisdell, 
Sawa, Leising, & Waldmann, 2006; Leising, Wong, Wald- 
mann, & Blaisdell, 2008). 
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Modeling causal queries Most studies on causal reasoning 
and learning have addressed how people make predic- 
tive inferences (from causes to effects) or assessments of 
causal strength. Causal knowledge representations, how- 
ever, also allow for other types of queries. More recently, 
psychologists have developed computational models 
of various types of queries and have empirically tested 
these models. These studies are again a test case for nor- 
mative theories that model how causal queries should 
be answered. For example, a set of studies on diagnos- 
tic reasoning by Meder et al. (2014) shows how devia- 
tions from previously held norms can lead to a modified 
normative theory that is also descriptively adequate (for 
overviews, see Meder & Mayrhofer, 2017; chapter 7.3 by 
Meder & Mayrhofer, this handbook). 

Meder et al. (2014) discovered that participants who 
were asked to make a diagnostic judgment did not sim- 
ply report the empirical probability of the cause given the 
effect, which has been considered the standard norma- 
tive measure of diagnostic probability, but decrease the 
diagnostic estimates when causal strength in the predic- 
tive direction is low. Meder and colleagues showed that 
this behavior is in fact rational under a Bayesian inference 
view. They developed a modified Bayesian model that is 
not only sensitive to the observed diagnostic probability 
but also incorporates the degree of uncertainty about the 
underlying causal model in the judgments. 

Other studies have addressed the question of how 
people make singular causal judgments (Cheng & Novick, 
2005; Stephan & Waldmann, 2018; Stephan, Mayrhofer, 
& Waldmann, 2020). For example, it may have been 
observed that Peter, a heavy smoker, has contracted lung 
disease. It is known that generally heavy smokers tend to 
contract lung disease later in their lives. A singular query 
might in this case ask whether Peter’s smoking actually 
caused his lung disease, in contrast to the possibility that 
in his case, the observed events are just a coincidence. 
The new model, which is again derived from normative 
accounts, tries to explain how people respond to such 
queries. 


Acquisition of causal model knowledge One of the major 
engineering achievements of Bayes net research in com- 
puter science are algorithms that allow machines to 
induce causal networks from covariation data using min- 
imal prior knowledge (e.g., constraint-based algorithms, 
Bayesian inference methods; Pearl, 2000; Spirtes et al., 
2001). In psychology, there has been a debate whether 
these algorithms are plausible models of human learn- 
ing (see Gopnik et al., 2004; Griffiths & Tenenbaum, 
2009). One problem is that these algorithms typically 
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require data sets of a size that clearly surpasses what 
human learners can process. Also, it is unclear whether 
humans are capable of making, without the aid of com- 
puters, the required precise estimates about conditional 
independence relations. Nevertheless, some psycholo- 
gists have tried to test whether human subjects are capa- 
ble of inducing causal models from covariation alone. In 
general, relatively simple tasks were presented in which 
a restricted set of causal models with a small number 
of variables (typically 3) were offered as candidates 
between which subjects were asked to choose. Neverthe- 
less, performance was generally poor (Steyvers, Tenen- 
baum, Wagenmakers, & Blum, 2003). Some studies have 
shown that allowing subjects to intervene helps a little 
bit (Bramley, Lagnado, & Speekenbrink, 2015; Gopnik 
et al., 2004; Steyvers et al., 2003). The general conclu- 
sion from these findings is that in principle, people can 
induce causal models from covariation data, but perfor- 
mance is poor unless learning conditions are extremely 
favorable. This is an example of research showing devia- 
tions from what normative models can achieve. 

The implausibility of domain-general algorithms of 
causal structure induction has led Waldmann (1996) to 
propose the view that people generally use prior knowl- 
edge about the structure of causal models to guide learning 
in a top-down fashion (“knowledge-based causal induc- 
tion”; see also Lagnado, Waldmann, Hagmayer, & Sloman, 
2007). Various cues, including temporal order, interven- 
tions, instructions, and prior knowledge, can guide the 
initial hypotheses about causal structure. Numerous stud- 
ies have shown that learners are capable of using these 
cues and are able to prioritize them in cases of conflict. 

Griffiths and Tenenbaum (2009) have proposed hierar- 
chical Bayesian inference strategies to model knowledge- 
or theory-based causal induction. The central assumption 
is that probabilistic inference is carried out at multiple 
levels of abstraction, which influence each other and are 
updated simultaneously. In causal learning, these lev- 
els include the data, alternative causal models, and the 
theory level, which may, for example, encode physical 
domain knowledge. Hierarchical Bayesian models speed 
up learning because the range of alternative hypotheses 
being considered is constrained by both the data and the 
theory level. 

Whereas theories combining top-down knowledge 
with bottom-up learning have initially focused on 
fairly abstract causal knowledge, such as knowledge 
about causal directionality, more recent accounts have 
incorporated specific domain knowledge. For example, 
Waldmann (2007) has presented evidence showing that 
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domain knowledge about different types of interactions 
of physical quantities influences the functional form of 
how multiple causes of a common effect are assumed to 
combine (see also Griffiths & Tenenbaum, 2009). 
Following up on earlier work demonstrating the 
role of temporal assumptions about cause-effect laten- 
cies (Buehner & May, 2003; Hagmayer & Waldmann, 
2002), more sophisticated models of the influence of 
time on causal induction have been proposed (Bramley, 
Gerstenberg, Mayrhofer, & Lagnado, 2018; Stephan et al., 
2020). Other approaches, postulating an integration of 
intuitive Newtonian physics with probabilistic infer- 
ence in causal models (“noisy Newton”), were offered 
by Gerstenberg and Tenenbaum (2017) and Sanborn, 
Mansinghka, and Griffiths (2013; see also Bramley, 
Gerstenberg, Tenenbaum, & Gureckis, 2018). 


Violations of Bayes net predictions Research about causal 
Bayes nets has initially found a surprisingly good fit 
between their theoretical assumptions and human 
causal inference (see Rottman & Hastie, 2014). However, 
more recently, deviations have been discovered, which 
led to attempts to modify the theory. 

Diagnostic reasoning was already discussed above as 
an example of how deviations from the predictions of 
normative theories may lead to a revised account that 
claims to be a better theory of diagnostic reasoning, both 
descriptively and normatively. Another popular topic 
questioning Bayes nets as a psychological account are 
apparent Markov violations, first discovered by Rehder 
and Burnett (2005). They showed, for example, that sub- 
jects who were presented with a common-cause model 
(see figure 7.2.1b) and then were asked to infer the prob- 
ability of an effect given its cause were influenced by 
how many other effects of this cause were present or 
absent. Prima facie, this finding violates the Markov con- 
straint, which is assumed to be a central feature of Bayes 
nets. According to this constraint, the requested cause-— 
effect inference should be independent of whether col- 
lateral effects are present or absent. 

Whereas some philosophers use this kind of evidence 
as a demonstration of the inadequacy of Bayes nets (e.g., 
Cartwright, 2001), cognitive psychologists chose a simi- 
lar strategy as Meder et al. (2014) and tried to show that 
Markov violations are only apparent violations of the 
normative account. They proposed extended Bayes net 
models with hidden variables, which explain the appar- 
ent Markov violations within normative Bayes nets that 
honor the Markov condition. Various theories have 
been proposed to motivate these extensions, for exam- 
ple, by postulating additional mechanism knowledge 
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(Park & Sloman, 2013) or intuitions about dispositions 
(Mayrhofer & Waldmann, 2015). Other accounts com- 
bine Bayes net representations with associative processes 
(Rehder, 2014; Rehder & Waldmann, 2017). 


3.2 The Disposition Framework 

A completely different view answers the question of 
why an observed causal relation between events holds 
by focusing on the participants involved in the causal 
interaction. An often-studied example are the two ani- 
mated colliding billiard balls in Michotte’s (1963) task. 

The normative status of dispositional theories can 
be debated. Some specific accounts were motivated by 
Aristotelian philosophy, which for centuries was viewed 
as rational but has been demonstrated to contradict 
modern physics (White, 2006). More recently, however, 
versions of dispositional theories have been developed 
that claim that these theories provide a better account 
of rational scientific research than competing views in 
philosophy of science (Cartwright & Pemberton, 2013; 
Mumford & Anjum, 2011). 

One important difference between dependency and 
dispositional theories concerns the causal relata. Whereas 
dependency theories focus on variables that, for instance, 
encode the presence or absence of events, dispositional 
theories use objects as primary entities. These objects may 
be humans or nonhuman entities. The dispositions of 
objects can be static (e.g., the solubility of sugar) or they 
can be transient and dynamic such as the sudden exer- 
tion of force when moving an object. Causal relations 
between events arise when objects are placed in specific 
situational contexts allowing them to express their dispo- 
sitions or powers. Thus, dispositional theories are looking 
for deeper explanations of observed dependencies under- 
lying the observed covariation. One can view this as a 
focus on underlying mechanisms; however, the mecha- 
nisms have different properties from mechanisms mod- 
eled within the dependency framework (e.g., as chains 
or networks of variables; see Stephan, Tentori, Pighin, & 
Waldmann, 2021). 

Dispositional theories can refer to very specific dispo- 
sitions (e.g., of aspirin; see Mumford & Anjum, 2011), 
but the accounts most popular in psychology typically 
distinguish between just two types of objects, often 
called causal agents and causal patients. A popular theory, 
which was initially developed in linguistic semantics, is 
force dynamics (see Gardenfors, 2014; Talmy, 1988). For 
example, Gardenfors (2014) analyzes causal scenarios 
as interactions between a causal agent, endowed with a 
force, and a causal patient, which can be an animate or 
inanimate, concrete or abstract object. The patient can 
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carry a counterforce resisting the action of the agent. 
Forces are primarily physical, but they can be extended 
metaphorically to social or mental forces (e.g., threats, 
commands, persuasions). Force dynamics has been used 
in linguistics to characterize verb semantics and argu- 
ment structure. For example, in “Peter hits Mary,” “hit” 
has two arguments, one describing an agent (Peter), the 
other the patient (Mary). 

Wolff (2007) has developed psychological variants of 
force dynamics (see also Wolff, Barbey, & Hausknecht, 
2010; for an overview, see Wolff & Thorstad, 2017). His 
force theory states that people evaluate configurations 
of forces attached to what Wolff calls “affectors” (i.e., 
agents) and “patients,” which may vary in direction and 
degree with respect to an end state. 


Representative research topics Numerous studies on 
force dynamics have been conducted in the context of 
linguistic semantics. My focus here will be on selected 
findings from psychology. 


Understanding causal terms Wolff (2007) used force the- 
ory to analyze the meaning of abstract causal concepts, 
such as cause, prevent, enable, and despite (see also Wolff 
& Song, 2003). For example, when we say, “High winds 
caused the man to move toward the bench,” we mean 
that the patient (the man) had no tendency to move 
toward the bench, the affector (i.e., agent; the wind) acted 
against the patient, and the resultant of the forces acting 
on the patient was directed toward the result of moving 
toward the bench. This constellation of forces describes 
how we understand the term cause. Related analyses 
were proposed for the other abstract causal terms. Wolff 
and colleagues tested their theory, for example, by pre- 
senting visual scenes to subjects depicting stationary or 
moving people or objects and asking subjects to choose 
between alternative causal terms (see Wolff, 2007; Wolff & 
Thorstad, 2017). 

Initially, the theory was applied to individual causal 
relations between agents and patients. Later, the theory 
was extended to more complex scenarios, such as causal 
chains or complex preventive relations, such as double 
preventions. For example, consider somebody hitting a 
pole that prevents a tent from falling. The action (hit- 
ting the pole) in this case prevents a preventer (the pole 
keeping the tent upright), which leads to the falling of 
the tent. Similarly, causation by omission can be mod- 
eled by considering the forces that are in play when an 
action is not executed (see Wolff et al., 2010; Wolff & 
Barbey, 2015). 


Causal asymmetry Another example of a dispositional 
account (although different from force dynamics) is 


White’s (2006) theory. Many of White’s studies have 
focused on the Michotte (1963) task, in which subjects 
observe two-dimensionally rendered animated moving 
balls. In a launching scenario, for example, object X, a 
ball, moves toward a resting object Y, another ball, and 
touches it. At this moment, object X stops and object 
Y begins moving, eliciting a causal impression. White 
(2009) has shown that subjects tend to view the launch- 
ing ball (X) as the agent and the launched ball (Y) as 
the patient (or “cause” and “effect object,” in his ter- 
minology). Subjects typically describe the launching 
scene as an event in which “X launched Y” instead of 
using the equally valid description that “Y stopped X.” 
Moreover, force estimates for X tend to be higher than 
force estimates for Y. According to White, both findings 
are indicators of the underlying dispositional distinc- 
tion between the two types of objects. Causal asymme- 
try contradicts Newtonian physics because the physical 
force on object Y exerted by object X is equal in mag- 
nitude (but opposite in direction) to that on object X 
exerted by object Y. From a Newtonian perspective, the 
collision is perfectly symmetric, and both descriptions 
(i.e., “X launched Y” and “Y stopped X”) should be 
equally appropriate (see also Mayrhofer & Waldmann, 
2014, 2016, for follow-up studies). 

White (2006, 2009) has proposed a dispositional the- 
ory of causal asymmetry that links perceived scenes to 
stored representations of sensomotoric experiences of 
our actions on objects. According to White, we expe- 
rience our own agency along with its associated forces 
during the course of our ontogenetic development. 
When we perceive a scene, we compare the movements 
of the objects with these stored representations. We tend 
to overestimate the force of the causal agent (i.e., cause 
object) relative to the counterforce of the manipulated 
patient (i.e., effect object) because this asymmetry corre- 
sponds to our experience of resistance that is overcome 
by our action. 


4. Conclusion 


Despite many attempts to develop a unitary theory of 
causal reasoning, this review of the literature shows 
that fundamentally different frameworks compete. Each 
framework has philosophical precursors that provide 
an initial normative basis, but within each framework, 
these accounts have been further developed with the 
goal to provide both descriptively more adequate theo- 
ries and, at least in some cases, also improved norma- 
tive accounts. Given that both the descriptive and the 
normative theories developed within and across the 
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frameworks compete, no uniform normative or descrip- 
tive theory of causal cognition has emerged. 

The frameworks differ in terms of the causal relata 
they employ and the way causal relations are construed. 
These differences make them more or less suitable for 
modeling different tasks. For example, dependency 
theories are particularly good at modeling how people 
predict events within complex causal models, whereas 
dispositional theories are typically applied to linguistic 
phrases or visual scenarios involving a limited number 
of participants. These specializations may be the reason 
why the different frameworks rarely compete directly 
with each other. Most of the debates in the causal cogni- 
tion literature concern competing theories within each 
framework. There are some limited attempts to cross the 
aisle and develop unitary frameworks that also apply to 
the tasks of the competitor (Cheng, 1993; Wolff, 2014), 
but these attempts have so far not been widely adopted 
by the research community. 

AS a consequence, other proposals have been pre- 
sented, which try to address the issue of competing frame- 
works. One position is causal pluralism, which accepts that 
different tasks may be modeled best by different types of 
theories (e.g., Lombrozo, 2010). Another approach is to 
develop hybrid theories that focus on how different kinds 
of causal representations interact in specific tasks. An 
example of a hybrid theory that combines dependency 
with dispositional representations has been presented by 
Mayrhofer and Waldmann (2015; see also Waldmann & 
Mayrhofer, 2016). They claim that verbal instructions, 
whose causal content can be best modeled by disposi- 
tional accounts, influence how causal Bayes nets are con- 
strued that are used to make causal inferences. These are 
just initial attempts. Future research will have to address 
in greater detail whether unitary, pluralistic, or hybrid 
accounts are best suited to model causal cognition. 
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7.3 Diagnostic Causal Reasoning 


Bjorn Meder and Ralf Mayrhofer 


Summary 


Rational theories of diagnostic reasoning assume that 
the reasoner’s goal is to infer the conditional probabil- 
ity of a cause given an effect from the available data. 
Typically, diagnostic reasoning is modeled within a sta- 
tistical inference framework, with Bayes’ rule applied to 
the obtained covariation information serving as the nor- 
mative standard. This chapter analyzes diagnostic rea- 
soning from the perspective of causal induction, using 
the framework of causal Bayes net theory to instanti- 
ate different accounts of rational diagnostic reasoning. 
These approaches elucidate the relevant kinds of inputs, 
computations, and outputs by differentiating between 
parametric causal models and observable contingency 
information. A particularly interesting feature of these 
accounts is that they can include predictions that sys- 
tematically deviate from the traditional, purely statistical 
norm. The analyses highlight key issues for constructing 
a rational theory of diagnostic reasoning and the experi- 
mental study of human rationality. 


1. Reasoning from Effect to Cause 


Diagnostic causal reasoning pertains to inferences from 
effect to cause, such as reasoning from symptoms to 
diseases in medical diagnosis. Probabilistic diagnostic 
inferences can be considered a special case of Bayesian 
inference, where beliefs about unobserved states of the 
world (the cause events) are updated in light of observed 
data (the effect events). The question to what extent 
people can make appropriate diagnostic inferences has 
been central to debates on human rationality, with prob- 
ability theory, and Bayes’ rule in particular, serving as 
a normative or descriptive reference point (Gigerenzer, 
1996; Kahneman & Tversky, 1996). Recent advances 
in causal modeling have provided new insights, with 
respect to both long-standing norms of rationality and 
the descriptive adequacy of models of human diagnostic 
reasoning. Causal Bayes net theory, that is, probabilistic 


inference over causal graphical models, provides a for- 
mal framework for representing causal dependencies and 
modeling different kinds of probabilistic causal infer- 
ences (Pearl, 2000; Spirtes, Glymour, & Scheines, 1993; 
Spohn, 1976/1978, as cited in Spohn, 2001; see also 
chapter 4.2 by Hartmann, this handbook). The frame- 
work explicates the relations between observed covaria- 
tion information and an underlying causal model that 
generates the data. Importantly, causal Bayes net theory 
has the expressive power to instantiate different ratio- 
nal models of diagnostic inference, thereby contesting 
the idea of a single normative benchmark for evaluating 
the rationality of human diagnostic reasoning (Meder 
& Mayrhofer, 2017b; Meder, Mayrhofer, & Waldmann, 
2014; Waldmann, Cheng, Hagmayer, & Blaisdell, 2008). 
This chapter highlights the ways in which a causal analy- 
sis of diagnostic reasoning can inform issues of theoreti- 
cal rationality and guide empirical research. 


2. Rational Models of Diagnostic Reasoning 


Different models of diagnostic reasoning have been 
postulated that serve the dual purpose of providing nor- 
mative standards and constituting candidate models of 
human cognition. Common to these approaches is the 
goal to infer the conditional probability of a cause given 
an effect. They critically differ, however, with respect to 
their assumptions regarding the relation between unob- 
servable causal structures and observable covariation 
information, as well as the involved representations and 
computations. 


2.1 Diagnostic Reasoning as Statistical Inference 

The most elemental form of diagnostic reasoning involves 
inferences from a binary (present vs. absent) effect event 
E to a binary cause event C. For instance, the cause event 
C could be a disease, with c and -c denoting the pres- 
ence and absence, respectively, of the disease, and the 
effect event E could be a particular symptom, with e and 
=e denoting the presence and absence, respectively, of 
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(a) Data samples 


(b) Causal structures 
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(c) Model predictions and human judgments 
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(a) Three joint frequency distributions over cause C and effect E. In each data sample, the diagnostic probability of cause C given 


effect E is 0.75. (b) Alternative causal structures that may have generated the data. According to structure 5S), there is a causal relation 


between C and E, as well as background causes A that can cause E independently of C. According to structure So, candidate cause C 


and effect E are independent; any observed empirical contingency is merely coincidental and caused by background causes A. 


(c) Model predictions and empirical results. The empirical probability of cause given effect is 0.75 in all three data samples. If uncer- 


tainty about possible causal structures is taken into account, different diagnostic probabilities result, depending on the extent to 


which the data warrant the existence of a causal relation (i-e., supports S; over So). Mean human judgments are not invariant across 


the three data sets, suggesting that people are sensitive to uncertainty about causal structure (Meder et al., 2014, experiment 2). 


the symptom. Here, we consider the situation where the 
diagnostic inference is based on a sample of observed 
data. For instance, a doctor may have data on how many 
patients with the disease have a particular symptom, 
how many without the disease have the symptom, and 
so on. Given this covariation information, the diagnostic 
probability of cause given effect, P(c|e), can be inferred 
using Bayes’ rule: 


P(e|c)-P(c) 
P(e|c)- P(c) + P(e| ac) - P(-c) 
_ Pelco): P(c) 
~ Pe)’ 


P(cle) 


(1) 


where P(c) denotes the prior probability (base rate) of the 
cause, P(e | c) is the likelihood of the effect given that the 
cause is present, and P(e | =c) is the likelihood of the effect 
if the cause is absent. Consider the data sample shown in 
figure 7.3.1a (top). According to these empirical frequen- 
cies, P(c) = 20/40 = 0.5, P(e|c) = 6/20 = 0.3, and P(e|-=c) = 
2/20 = 0.1. Plugging these values into equation (1) yields 
the diagnostic probability P(c |e) = 0.75. 

Whether people reason in line with Bayes’ rule has 
been pivotal to disputes about human rationality. Studies 
in the 1950s and 1960s indicated that subjects updated 
their beliefs to a lesser extent than prescribed by Bayes’ 


rule (Edwards, 1968; Phillips & Edwards, 1966). By and 
large, though, the experimental findings were considered 
evidence for sound probabilistic reasoning, giving rise to 
the metaphor of “man as intuitive statistician” (Peterson 
& Beach, 1967). Researchers in the heuristics-and-biases 
program, however, came to a rather different conclusion, 
arguing that typically people’s diagnostic inferences are 
not in line with classic Bayesian norms (e.g., Kahneman, 
Slovic, & Tversky, 1982; Tversky & Kahneman, 1974; 
for a critical review, see Koehler, 1996). The discrepancy 
has led researchers to more precisely characterize the 
conditions under which people can make appropriate 
diagnostic inferences, for instance, when probabilistic 
information is conveyed in particular frequency formats 
(Gigerenzer & Hoffrage, 1995; McDowell & Jacobs, 2017; 
Meder & Gigerenzer, 2014). 

The conditional probability of cause given effect 
derived by applying Bayes’ rule to verbally described 
probabilities or sample data has been endorsed by many 
researchers as the normative standard, even when they 
otherwise debate whether and under what conditions 
people can solve such tasks. Next, we analyze diagnos- 
tic reasoning from the perspective of inductive causal 
inferences, using causal Bayes net theory to implement 
different candidate models for a rational account of 
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diagnostic reasoning. These approaches differentiate 
between parametric causal models and observable con- 
tingency information and can therefore lead to very 
different predictions than accounts that derive the con- 
ditional probability of cause given effect directly from 
the observed empirical data without any reference to an 
underlying causal model. 


2.2 Diagnostic Reasoning as Causal Inference 

One goal of rational agents is to acquire knowledge about 
the causal structure of noisy environments, in order to 
support prediction, diagnosis, and control (Waldmann, 
2017; chapter 7.2 by Waldmann, this handbook). In this 
view, diagnostic inferences operate on representations 
that preserve the directionality of causal relations (as 
opposed to undirected statistical or associative relations). 
This distinguishes the account from Bayes’ rule, which 
is applicable to arbitrary statistically related events and 
does not make any reference to possible causal relations 
that may underlie the observed data. 

The standard causal model for situations involving a 
binary cause event and a binary effect event is structure 
S; in figure 7.3.1b. This graph states that there is a causal 
relation between C and E, as well as an amalgam of unob- 
served background causes A that occur independently of 
C and can also generate E. Cheng (1997; see also Glym- 
our, 2003; Griffiths & Tenenbaum, 2005; Novick & 
Cheng, 2004) showed that the generative causal power of 
a cause—the unobservable probability with which C pro- 
duces E—can be estimated according to 

reps CALs ees (2) 

1- P(e|-=c) 
where w, is the causal power of C with respect to E. 
The diagnostic probability of cause given effect can be 
inferred from the parameterized causal structure: 


Ple|c)- P(c) 
P(e|c)- P(c) + P(e|=c) + P(-c) 
_ wb, + w,b. — w.w,b, 
w.b. + W, — W.W, Db. 


P(cle)= 
(3) 


where w, is the causal power of C with respect to E, b. 
denotes the base rate (prior probability) of the cause, and 
Wa denotes the strength of the background cause A. 

The quantities used to parameterize the causal struc- 
ture are typically single-point estimates derived directly 
from the empirical frequencies (i.e., maximum likelihood 
estimates of P(c), P(e|c), P(e| 7c), and, hence, w.; see, e.g., 
Cheng, 1997). In this case, the conditional probability of 
cause given effect derived from the parameterized causal 
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graph exactly corresponds to the values obtained from 
directly applying Bayes’ rule to the empirical frequencies, 
although diagnostic reasoning takes place on the causal 
rather than the data level (Meder et al., 2014).! This is 
not necessarily the case, though, if uncertainty about 
causal parameters or alternative generative causal models 
is incorporated in the diagnostic inference process. 


2.3 Diagnostic Reasoning: Parameter Uncertainty 

and Causal Structure Uncertainty 

Analyzing diagnostic reasoning from the perspective 
of causal induction enables consideration of different 
kinds of uncertainty in the inference process. Parame- 
ter uncertainty arises if estimates are derived from limited 
and potentially noisy data. Formally, this type of uncer- 
tainty can be modeled by using probability distributions 
for representing causal parameters, rather than single- 
point estimates, with the parameter distributions being 
updated in light of the data using Bayesian inference 
(Lu, Yuille, Liljeholm, Cheng, & Holyoak, 2008). Impor- 
tantly, the posterior distributions quantify and represent 
uncertainty explicitly (e.g., through the variance of the 
distributions). 

Another type of uncertainty is structure uncertainty, 
which pertains to possible causal models that may 
underlie the observed data. The structure induction model 
of diagnostic reasoning (Meder et al., 2014) takes this into 
account by considering two causal graphs (figure 7.3.1b): 
instead of using only the default structure S,, the model 
also considers an alternative structure So according to 
which there is no causal relation between C and E. The 
intuition behind this is that an observed contingency 
between C and E may not be indicative of a causal rela- 
tion but merely coincidental, and rational agents should 
take this into account. The data are then used to esti- 
mate the causal structures’ parameters through Bayesian 
inference, and under each parameterized structure, an 
estimate of the diagnostic probability is derived. Under 
Si, assuming uniform prior distributions over the par- 
ameters, the computed probability will approximate the 
empirically observed P(c|e). Structure So, by contrast, 
states that C and E are independent events; therefore, 
observing E does not provide diagnostic evidence for C 
(i.e., P(c| e) = P(c)). The structure induction model com- 
putes how likely each of the two structures is, given the 
data, and forms a weighted average of the entailed diag- 
nostic probabilities, with the resulting estimate taking 
into account both parameter and structure uncertainty. 

Depending on the relative probability of Sy and S;, the 
inferred diagnostic probability can strongly diverge from 
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the empirical probability. Figure 7.3.1c illustrates this for 
the three data sets shown in figure 7.3.1a: in all three 
data sets, the empirical probability is P(c|e) = 0.75; there- 
fore, a rational agent might conclude that the diagnostic 
probability of cause given effect is the same in all three 
situations. Consideration of alternative causal structures 
leads to very different inferences. For instance, for sam- 
ple 1, the structure induction model entails a diagnostic 
probability of 0.61, much lower than the empirical prob- 
ability of 0.75. This discrepancy arises from the fact that 
the contingency between cause and effect is relatively 
weak, so that structures So and S; are almost equally likely 
to have generated the data. For samples 2 and 3, the dis- 
crepancy is smaller, as these data indicate a stronger con- 
tingency between C and E, making it more likely that S; 
is indeed the true generating model. 

Interestingly, human diagnostic judgments mirror 
structure uncertainty: judgments strongly vary, although 
the empirical probability of cause given effect is identi- 
cal across the three data sets (figure 7.3.1c; for details, see 
Meder et al., 2014). From the perspective of the classic sta- 
tistical inference perspective, this response pattern looks 
irrational. By contrast, viewed as resulting from a causal 
inference strategy that is adapted to the uncertainties of 
the world outside the laboratory, the judgments reflect 
rational reasoning. 


3. Rational Models of Causal Attribution 


Theories of diagnostic reasoning typically assume that the 
goal of the reasoner is to infer the conditional probability 
of cause given effect. In many diagnostic reasoning sce- 
narios, however, it is judgments of the causal responsibility 
(or causal attribution) that are of interest, that is, judg- 
ments of the probability that a candidate cause brought 
about the effect. For instance, instead of merely assessing 
the probability of a particular genetic disposition’s being 
present, a doctor may want to find out whether that dis- 
position is the cause of a patient’s symptoms. This quan- 
tity is different from the diagnostic probability of cause 
given effect. For instance, if there is no causal relation 
(i.e., P(e|c) = P(e|-c) and, therefore, w, = 0), it holds that 
P(c|e) = P(o). But, intuitively, if there is no causal depen- 
dency, then the probability that C produced E is zero. 
Whereas a purely statistical account lacks the expres- 
sive power to model judgments of causal attribu- 
tion, they can be formalized within a causal modeling 
approach (Cheng & Novick, 2005). Let ce denote that 
the presence of effect E is generated by the presence of 
cause C. The query of whether the occurrence of effect 
E can be attributed to cause C translates to determining 
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the conditional probability P(c—e|e). Given parame- 
terized structure S;, this quantity can be computed as 
follows (for details and for further measures of causal 
responsibility, see Cheng & Novick, 2005): 


P(e|ce): P(ce) 
P(e) 
bw. 

b.w. + w, — b.w.w, * 


P(coe|e) = 
(4) 


In its original formulation, parameters b., w,, and w-, 
are maximum likelihood point estimates directly derived 
from the empirical data.” Bayesian variants of the model 
(Holyoak, Lee, & Lu, 2010) can incorporate parameter 
uncertainty, and the computations can also be incorpo- 
rated into the structure induction model of diagnostic 
reasoning, in which case estimates of causal responsi- 
bility take into account both parameter and structure 
uncertainty (Meder et al., 2014). This approach has been 
successfully used to account for people’s judgments of 
causal attribution about singular cases (Stephan & Wald- 
mann, 2016, 2018). 


4. Diagnostic Reasoning in Complex 
Causal Networks 


Real-world scenarios typically involve complex causal 
networks relating multiple causes and multiple effects, 
such as diagnostic reasoning with multiple symptoms 
and multiple possible diseases. In the causal Bayes net 
framework, the factorization of the joint probability dis- 
tribution over the considered variables is determined 
by the causal relations in the graph. This follows from 
applying the causal Markov condition, according to which 
the state of any variable in the graph is a function only 
of its direct causes, rendering it independent of all other 
variables except its direct and indirect effects (Hausman 
& Woodward, 1999; for a critique, see Cartwright, 1989, 
1999). For instance, in the common-effect model shown 
in figure 7.3.2a, causes A and C are unconditionally inde- 
pendent but dependent conditional on their common 
effect B. In the common-cause model (figure 7.3.2b), A 
and C are two effects of acommon cause B. If the Markov 
condition holds, this implies that A and C are uncondi- 
tionally dependent but independent conditional on their 
common cause B (B “screens off” the correlation between 
A and C; Reichenbach, 1956). Similarly, in the causal- 
chain model (figure 7.3.2c), A and C are unconditionally 
dependent but conditionally independent given B. 

The causality-based factorization implies particular 
dependence and independence relations among the vari- 
ables, which facilitate and constrain diagnostic inferences 
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(a) Common-effect structure 


P(A, B, C) = P(A): P(C): P(B|A,C) 


Figure 7.3.2 


(b) Common-cause structure 


P(A, B, C) = P(B): P(A|B): P(C| B) 
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(c) Causal-chain structure 


P(A, B, C) = P(A): P(B|A)- P(C|B) 


Basic causal structures comprising three variables A, B, and C. Applying the causal Markov condition to each causal graph entails 


a different factorization of the joint probability distribution P(A, B,C) over domain variables A, B, and C. 


across causal networks. For instance, when reasoning diag- 
nostically in common-effect structures, explaining away is 
an intriguing inference pattern (Morris & Larrick, 1995; 
Pearl, 1988). Explaining away occurs when conditioning 
not only on the effect but also on the known presence of 
an alternative cause. Consider figure 7.3.2a and assume 
B is a symptom (e.g., fever) with two independent, not 
mutually exclusive, causes A and C (e.g., a virus and a bac- 
terial infection). With respect to cause A, explaining away 
corresponds to the inequality P(a|b) > P(a| b,c). Knowing 
that a patient has a fever will raise the probability of her 
being infected with the virus, but learning additionally 
that she has a bacterial infection will lower the probability 
of the virus to some extent. In other words, knowing that 
not only effect B but also the alternative cause C is present 
will explain away some of the diagnostic evidence pro- 
vided by B with respect to A (for an overview, see Khem- 
lani & Oppenheimer, 2011). 

Another kind of diagnostic reasoning involves infer- 
ences from multiple effects (e.g., different symptoms) toan 
underlying cause (e.g., a disease), constituting a common- 
cause structure (figure 7.3.2b; Meder & Mayrhofer, 
2017a). According to this structure, effects A and C are 
unconditionally dependent (e.g., lung cancer and yellow 
teeth correlate because of their common cause, smoking) 
but independent conditional on their common cause B. 
This property strongly simplifies diagnostic inferences, 
because the number of estimates required to parameterize 
the causal structure is greatly reduced. In particular, the 
joint likelihood of the effects given the cause can be com- 
puted as the product of the individual likelihoods of the 
effects given the cause. However, research also indicates 
that people’s judgments do not always honor the causal 
Markov condition, suggesting important pathways for 
further investigating the rationality of human reasoning 
(Mayrhofer & Waldmann, 2016; Park & Sloman, 2013; 
Rottman & Hastie, 2016). 


5. Toward a Rational Theory 
of Diagnostic Reasoning 


Models of diagnostic reasoning fundamentally differ 
with respect to their assumptions, representations, and 
involved computations. As a consequence, they can 
make strongly diverging predictions in many situations. 
What are the implications for constructing a rational 
theory of diagnostic inference, and what does this mean 
for the experimental study of human rationality? 

On the theoretical level, analyzing diagnostic reason- 
ing from the perspective of inductive causal inference 
provides new insights into an old problem. Causal Bayes 
net theories have the expressive power to distinguish 
between the data level (i.e., covariation information) and 
parametric causal models that could underlie the obser- 
vations. A causal modeling framework also enables for- 
malizing diagnostic inferences that do not pertain to the 
conditional probability of cause given effect, such as judg- 
ments of causal responsibility. Some of the accounts that 
can be implemented within this framework are isomor- 
phic to the classic, purely statistical model in the sense 
that the inferred diagnostic probability of cause given 
effect is identical to applying Bayes’ rule to the empiri- 
cal probabilities. Other models make very different pre- 
dictions about what would constitute a rational solution 
to the inference problem, for instance, when incorporat- 
ing uncertainty about alternative causal structures into 
the judgment process. As a result, inferences that look 
irrational from the perspective of one model (e.g., a sta- 
tistical account operating on the data level) would be con- 
sidered rational from the perspective of another model 
(e.g., a causal inference account that considers structure 
uncertainty). 

This also raises critical methodological issues for the 
experimental study of human rationality, which all too 
often involves comparing human behavior to a single, 
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supposedly normative, yardstick. Rarely, if ever, will 
there be a single rational theory for a particular phenom- 
enon (Cohen, 1981; Gigerenzer, 1996; cf. chapter 2.4 by 
Fiedler, Prager, & McCaughey, this handbook). First, dif- 
ferent frameworks exist that can—and should—be used 
to construct rational theories, including probability the- 
ory (Anderson, 1990; chapter 4.5 by Chater & Oaksford, 
this handbook), logic-based theories (Ragni & Knauff, 
2013), ranking theory (Spohn, 2012; chapter 5.3 by Kern- 
Isberner, Skovgaard-Olsen, & Spohn, this handbook), and 
theories of bounded and ecological rationality (Chase, 
Hertwig, & Gigerenzer, 1998; chapter 8.5 by Hertwig & 
Kozyreva, this handbook). Comparing different types of 
models can provide insights that could not be gained 
when restricting the analysis to one particular theoretical 
viewpoint. Moreover, within a particular methodological 
framework, different models can be implemented and 
defended as rational, challenging the common approach 
of comparing human behavior to a single, supposedly 
normative, standard (Meder et al., 2014). In the case of ele- 
mental diagnostic reasoning, all the models discussed in 
this chapter rely on some form of probabilistic inference, 
but their scope, assumptions, and predictions strongly dif- 
fer. From the perspective of the behavioral sciences, these 
models should be considered candidate theories, not stan- 
dards, of human behavior (McKenzie, 2003). In this view, 
the influence runs both ways: rational theories can inform 
empirical research, but if there is a stable behavioral pat- 
tern that is inconsistent with a particular rational model, 
one should also revise one’s beliefs about the appropriate- 
ness of the presumed normative yardstick. 
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Notes 


1. Consider sample 1 in figure 7.3.1a (top). The empirical 
probabilities derived from the frequency data are P(c) =0.5, 
P(e|c) = 0.3, and P(e|-7c) = 0.1. Accordingly, w. = 0.22 (equa- 
tion (2)), with P(c) and P(e|-c) serving as estimates for b. and 
W,, respectively. Plugging these values into equation (3) yields 
P(c|e) = 0.75; the same value results when applying Bayes’ rule 
to the empirical probabilities (equation (1)). 


2. Consider again sample 1 in figure 7.3.1a (top). Using these 
data to parameterize structure S$, with maximum likelihood 
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point estimates yields b. = 0.5, w. = 0.22, and w, = 0.1. Plugging 
these values into equation (3) yields P(c|e) = 0.75, whereas the 
estimate for the probability that C produced E derived from 
equation (4) yields P(ce|e) = 0.56. 
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Part Ill Practical Rationality 


Section 8 Individual Rationality and Decision Making 


8.1 Preferences and Utility Functions 
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Summary 


This chapter discusses the notions of preference and 
utility in relation to individual judgment and decision 
making. The first two sections sketch the formal prop- 
erties of preferences—in particular, completeness and 
transitivity—and of utility functions—in particular, con- 
tinuity, differentiability, diminishing marginal utility, 
and risk attitudes. Whether these properties should be 
considered as conditions of practical rationality is also 
discussed in section 2. The third section focuses on the 
dominant interpretations of the formal preference and 
utility notions, in particular, distinguishing between 
mental and behavioral interpretations. The fourth sec- 
tion discusses relations between total and partial pref- 
erences, and the last section presents some models of 
rational preference change. 


1. Preference Relations 


The most common way to represent preferences is as 
a binary relation between two relata. This is how pref- 
erence is typically conceptualized both in the social 
sciences and in philosophy. The properties that this rela- 
tion is assumed to have are commonly described in a 
formalized language.' The preferences studied in prefer- 
ence logic are usually the preferences of rational individ- 
uals, but preference logic is also used in psychology and 
behavioral economics, where the emphasis is on actual 
preferences as revealed in behavior. 


1.1 Concepts and Notation 

The basic formal preference concept consists of a binary 
relation > on a set of alternatives A. This relation is most 
commonly used to denote the objectives of a decision 
maker as a comparative evaluation. That is, x = y compares 
two members x and y of A, describing x to be at least as good 
as y. This comparison might represent subjective desires 
of an individual but also her evaluative judgments. Fur- 
thermore, it might also represent evaluative judgments of 


social agents (e.g., groups, companies, government insti- 
tutions) or intersubjective evaluations that do not coin- 
cide with the subjective attitudes of any particular agent 
(e.g., certain types of moral judgments). Finally, prefer- 
ence relations are also used to represent choice patterns. 

The logical properties of this comparative relation do 
not seem to differ between the cases where they corre- 
spond to what we usually call “preferences” and the cases 
where they do not. The term “preference logic” is there- 
fore standardly used to analyze the basic properties of 
these relations irrespective of their interpretations. 

From the relation +, two further relations on A can be 
derived. The strict preference relation > is defined by 


x>y ifandonlyif xy andnot yx (1) 


nou 


and reads as “x is strictly preferred to y,” “x is better 
than y,” or “y is worse than x.” The indifference relation 
is defined by 


x~y ifandonlyif x=y and yex (2) 


and reads as “x is indifferent to y” or “x is equal in value 
to y” (von Wright, 1963). 

The above-described relations relate members of A, 
the set of alternatives. Most broadly, one can characterize 
Aas a set of propositions (Jeffrey, 1965/1983). Economists 
typically consider A to consist of vectors of consumption 
goods (Debreu, 1959). In most applications, members of 
Aare assumed to be mutually exclusive (i.e., none of them 
is compatible with, or included in, any of the others). Pref- 
erences over a set of mutually exclusive relata are referred 
to as exclusionary preferences (S. O. Hansson, 2001). 

The properties discussed in this section all concern such 
exclusionary preferences. In practice, people also have 
preferences between relata that are not mutually exclusive. 
These are called combinative preferences (S. O. Hansson, 
2001). 

An important kind of preferences are preferences over cer- 
tain outcomes, where the relata are conjunctions of propo- 
sitions that are interpreted as describing specific states of 
the world. Another kind are preferences over lotteries, where 
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the relata are mutually exclusive and jointly exhaustive 
disjuncts of propositions that are interpreted as alterna- 
tive states of the world, each of which might be realized 
with a certain probability. Many relata of actual prefer- 
ences do not satisfy either of these two conditions—in 
particular, the relevant disjuncts might be only partly 
identified, thus making it difficult to describe the relata 
as well-formed formulas. Such relata feature in decision 
making under deep uncertainty (S. O. Hansson, 1996). 


1.2 Completeness 

In most applications of preference logic, it is taken for 
granted that the following property, called “complete- 
ness” or “connectedness,” should be satisfied: 


Xe yvyex, forall x,yEA, (3) 


that is, = is assumed to connect every member of A to 
every other member. 

Completeness is assumed in many applications, not 
least in economics. Bayesian decision theory is a case in 
point. The Bayesian decision maker is assumed to make 
her choices in accordance with a complete preference 
ordering over the available options (Savage, 1954/1972, 
p. 18). However, in many everyday cases, we do not 
have, and do not need, complete preferences. Consider 
a person who has to choose between three objects a, b, 
and c. If she knows that she prefers a to the others, she 
does not have to make up her mind about the relative 
ranking among b and c. In such cases, preference incom- 
pleteness might be rationally justifiable. 

In many practical contexts, people might exhibit 
incomplete preferences, even if not rationally justified. 
This raises the question how incompleteness can be 
resolved, which might happen in different ways. First, 
if the reason for preference incompleteness is lack of 
knowledge or reflection of one’s own desires or judg- 
ments, then through observation, introspection, logical 
inference, or some other means of discovery, the missing 
preference relations might be discovered. In that case, 
the incompleteness is resolved in exactly one way, by 
arriving at exactly one complete preference relation. Sec- 
ond, incompleteness may be resolvable in several differ- 
ent ways. In this case, one’s own desires or judgments 
on these issues are genuinely undetermined, so that the 
missing preferences cannot be discovered but must be 
constructed (Slovic, 1995). Finally, incompleteness may 
be irresolvable. Even with all the necessary resources 
available for preference discovery or construction, a per- 
son might be unable to say which she prefers. This last 
case of preference incompleteness is often also discussed 
as “preference incommensurability” (Chang, 1997). 
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1.3 Transitivity 
By far the most discussed logical property of preferences 
is transitivity: 


XEVAVSZ—>xXezZ, forall x,y,zEXx. (4) 


That is, if x is weakly preferred to y, and y weakly preferred 
to z, then x is weakly preferred to z. Experiments suggest 
that intransitive preferences are quite common (e.g., Tver- 
sky, 1969). Nevertheless, there is a strong tradition, not 
least in economic applications, to regard >-transitivity 
as a necessary prerequisite of rationality. 

The most famous argument in favor of preference 
transitivity is the money-pump argument. Its basic idea is 
that transitivity ensures against practical losses (Davidson, 
McKinsey, & Suppes, 1955; Ramsey, 1928/1950). Consider 
a situation where a stamp collector has cyclic preferences 
with respect to three stamps, denoted x, y, and z. He prefers 
x toy, y to z, and z to x. He is willing to pay 10 cents every 
time he exchanges a stamp for one he prefers. Now he 
enters a stamp shop with stamp x. The stamp dealer offers 
him to trade in x for z if he pays 10 cents. He accepts the 
deal. Next, the dealer offers him to trade in z for y, again for 
a 10-cent fee, which he accepts. Finally, the dealer offers 
him to trade in y for x, again for a 10-cent fee, which he 
accepts. The collector now has obtained the same stamp 
with which he entered the shop but is 30 cents poorer. 
Unless he revises his preferences such that they satisfy (4), 
he will accept further trades from the dealer until he runs 
out of funds. The money-pump argument thus claims to 
show that transitivity is a necessary condition in order to 
avoid negative practical results. 

Various arguments have been raised against the money- 
pump justification of transitivity. First, it has been argued 
that it relies on particular conditions, which must be 
in place for intransitive preferences to be exploited in 
this way. In particular, it must be assumed that the thus- 
pumped individual cannot take recourse to some strategy 
that avoids his being pumped—for example, through pre- 
commitment or resolution (McClennen, 1990). Second, 
it has been argued that the money-pump argument itself 
must make relatively strong consistency assumptions, 
which undermine its generality (Cubitt & Sugden, 2001). 

Another argument for the normative appropriateness 
of preference transitivity suggests that transitivity is con- 
stitutive of the meaning of preference. Drawing an anal- 
ogy to length measurement, Davidson (1976/1980) asks, 
“Tf length is not transitive, what does it mean to use a 
number to measure length at all? We could find or invent 
an answer, but unless or until we do, we must strive to 
interpret ‘longer than’ so that it comes out transitive. 
Similarly for ‘preferred to’” (p. 273). Violating transitivity, 
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Davidson claims, thus undermines the very meaning of 
preferring one option over others. 

Yet another argument rests on the importance of pref- 
erences for choice. When agents choose simultaneously 
from all the elements of an alternative set, then prefer- 
ences should be choice-guiding. They should have such 
a structure that they can be used to guide their choice 
among the elements of that set. But when choosing, for 
example, from {x,y,z}, a preference relation > such that 
X>y>Z>x does not guide choice at all: any or none 
of the alternatives should be chosen according to >. 
The transitivity of preference, it is therefore suggested, 
is a necessary condition for a meaningful connection 
between preferences and choice. A critic, however, can 
point out that preferences are important even when they 
cannot guide choices. Take, for example, preferences over 
lottery outcomes: these are real preferences, regardless 
of the fact that one cannot choose between lottery out- 
comes. Furthermore, the indifference relation does not 
satisfy choice guidance either. That does not make it irra- 
tional to be indifferent between alternatives. Finally, the 
necessary criteria for choice guidance are much weaker 
than transitivity (S. O. Hansson, 2001, pp. 23-25). Thus, 
choice guidance can be an argument for the normative 
appropriateness of transitivity only under certain restric- 
tions, if at all (for further discussion, see Anand, 1993). 

All of these arguments for transitivity remain controver- 
sial. In addition, examples have been offered that suggest 
intransitive preferences to be rational at least sometimes. 
One important class of such counterexamples is derived 
from the classic sorites paradox. Perhaps the most famous 
example applied to preference transitivity is Quinn’s self- 
torturer. Here a device has been implanted into the body 
of a person. The device has 1,001 settings, from 0 (off) to 
1,000. Each increase leads to a negligible increase in pain. 
Each week, the self-torturer “has only two options—to 
stay put or to advance the dial one setting. But he may 
advance only one step each week, and he may never 
retreat. At each advance he gets $10,000.” In this way he 
may “eventually reach settings that will be so painful that 
he would then gladly relinquish his fortune and return 
to 0” (Quinn, 1990, p. 79). The self-torturer thus reveals 
intransitive preferences. But Quinn argued that these pref- 
erences are not irrational, because the self-torturer has good 
reasons to turn the dial, yet also has good reasons to want 
to revert to the original position. Much depends here on 
what might constitute good reasons to turn the dial. Some 
have argued that the self-torturer’s inability to distinguish 
between two adjacent dial settings might constitute such 
a reason (Luce, 1956). Others have sought to explain the 
intransitive preferences as the result of a heuristic decision 
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procedure (Voorhoeve & Binmore, 2006). It therefore 
seems that, ultimately, the rationality judgment depends 
on the process through which these preferences are gener- 
ated, and here considerable ambiguity remains. 

Another important class of counterexamples to tran- 
sitivity is derived from the classic Condorcet paradox. A 
simple example here is Schumm’s Christmas ornaments 
(Schumm, 1987), involving boxes containing three balls, 
colored red, blue, and green, respectively; they are repre- 
sented by the vectors (R1, G1, B1), (R2, G2, B2), and (R3, 
G3, B3). A chooser might strictly prefer box 1 to box 2, 
since they contain (to her) equally attractive blue and 
green balls, but the red ball of box 1 is more attractive 
than that of box 2. She prefers box 2 to box 3, since they 
are equal but for the green ball of box 2, which is more 
attractive than that of box 3. And finally, she prefers box 
3 to box 1, since they are equal but for the blue ball of 
box 3, which is more attractive than that of box 1. Thus, 


R1>R2~R3~R1, 

G1l~G2+ G3~ G1, 

B1~B2~B3>B1; and 

(R1,G1,B1) + (R2, G2, B2) > (R3, G3, B3) + (R1,G1,B1). 


The described situation yields a preference cycle, 
which contradicts transitivity of strict preference—yet 
arguably, each of these preferences is rational. 


1.4 Order Typology 

The completeness and transitivity of =, in combination 
with the definitions of > and ~, imply the following prop- 
erties (Mas-Colell, Whinston, & Green, 1995, p. 7): 


> is irreflexive: x > x never holds. (5) 
> is asymmetric: x > y — 7(z> x) for all x,y GA. (6) 


>is transitive: x>yny>z— x>Zz forall x,y,zEA. (7) 


> is acyclical: there is no series x1, ..., Xn 

of alternatives such that x1>... > Xn > 1. (8) 
~ is reflexive: x ~ x for all x GA. (9) 
~ is symmetric: x ~y — y~x for all x,y GA. (10) 
~ is transitive:-X~VANV~Z>X~Z 

for all x,y,z EA. (11) 
IP-transitivity: Xx ~y NY>Z—>X>Z 

for all x,y,z EA. (12) 
Pl-transitivity: x -VnV~Z>XxX>Zz 

for all x,y,z EA. (13) 
Weak connectivity of =: if x  y, then 

Xeyv yeX for all x,y,z EA. (14) 
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All of these properties individually or in conjunction 
constitute a weakening of the completeness and tran- 
sitivity of = (with the exception of those conjunctions 
that imply completeness or transitivity). Given the vari- 
ous counterarguments discussed above, some authors 
have suggested replacing completeness and transitivity 
with some of their weaker implications. 

Sometimes, the following property is added to the 
above list, even though it is not implied by the com- 
pleteness and transitivity of =: 


Antisymmetry of =: X=VA VEX >X=Y. (1S) 


Preference orderings of varying strength can be char- 
acterized by combinations of the above properties. Some 
of their common names are given in table 8.1.1 (Debreu, 
1959; Sen, 1970). 


2. Utility Functions 


Another way to represent value comparisons or choice 
patterns is via a real-valued function u: A > R, which 
maps the members of A into the real numbers. Of two 
alternatives x and y, we say “x is strictly preferred to y,” 
“x is better than y,” or “y is worse than x” if and only if 
u(x) > u(y), and “x is indifferent to y” or “x is equal in 
value to y” if and only if u(x) = u(y). 


2.1 Ordinal Scale Utility Functions 

Under this interpretation, there is an obvious connec- 
tion to the relational representation of preferences. 
Specifically, 


Xey ifandonlyif u(x) =u(y) 


1 
(Ordinal Representation). os 


Any function u that assigns a larger number to x than 
to y will work as such a representation. Consequently, 


Table 8.1.1 
An (incomplete) overview of ordering terminologies 


Properties Name(s) 


Reflexive, transitive Preorder, quasi-order 


2. Reflexive, transitive, Partial order 


antisymmetric 


Irreflexive, transitive Strict partial order 


4. Reflexive, transitive, complete Total preorder, complete 
quasi-ordering, weak 


ordering 


5. Reflexive, transitive, complete, 
antisymmetric 


Chain, linear ordering, 
complete ordering 


6. Asymmetric, transitive, weakly 
connected 


Strict total order, strong 
ordering 
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the function u can be replaced with any function u’ as 
long as u' is a positive monotone transformation of u. As 
this transformation property is the defining character- 
istic of ordinal scales, we call this an ordinal preference 
representation. 

However, a preference relation = can be ordinally 
represented by a utility function only if it is transitive 
and complete (von Neumann & Morgenstern, 1947). An 
incomplete preference ordering has an ordinal represen- 
tation that satisfies 


if x=y then u(x) = u(y). (17) 


The converse is obviously not true. However, incomplete 
preferences have been represented as sets of utility func- 
tions (Aumann, 1962). 

Not every transitive and complete preference rela- 
tion > can be ordinally represented by a utility function. 
Consider, for example, a lexicographic preference over 
goods bundles (x,y) ERxR with 


(X1,V1) = (X2, V2) iff x12xX2 Or (41 =X. and yizyr). 


Unless x;# x; for all goods bundles (xi, yi), (x, yj) with i#/j, 
such a relation > cannot be represented by a utility func- 
tion, although > is transitive and complete (Debreu, 1954). 

A sufficient condition for a transitive and complete 
preference relation = to be ordinally representable is the 
continuity of >: 


> is continuous iff for all x EX, the upper contour set 
of x, {vy EX: y= x}, and the lower contour set of x, {y EX: 
xX = y}, are closed, that is, they include their boundaries. 


Lexicographic preferences like the above example are 
not continuous. Every continuous, transitive, and com- 
plete preference relation > is ordinally representable by 
a utility function u (Mas-Colell et al., 1995, pp. 46-47). 

In the social sciences, utility functions are typically 
analyzed with the tools of maximization under con- 
straints. This requires that the thus-analyzed functions are 
twice differentiable. However, not every continuous, transi- 
tive, and complete preference relation > is representable 
by a differentiable utility function. Consider, for example, 
the preference (x1, 1) = (x2, y2) iff min{x1, yi} = min{x2, yo} 
(Leontief, 1941). 

Lexicographic and Leontief preferences appear at 
least sometimes to be fully rational. For example, it 
seems rational to lexicographically prefer subsistence 
goods to luxury goods when living at subsistence levels. 
Similarly, it seems rational to have Leontief preferences 
for perfectly complementary goods, like left and right 
shoes. Therefore, standard utility functions cannot ordi- 
nally represent all rational preferences. 
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2.2 Interval-Scale Utility Functions 
Although utility functions cannot represent all prefer- 
ences, an ordinal-scale functional representation of > rep- 
resents nothing but information contained in =. In other 
words, all information contained in an ordinal utility 
function can be represented by a (transitive and complete) 
ordering >. This is not the case if we interpret utility func- 
tions cardinally—that is, either as an interval scale or as a 
ratio scale. A utility function as an interval scale allows for 
degrees of difference between preferences, such that, for 
example, the utility assignments u(x) = 12, u(y) = 6, and 
u(z) = 3 represent the information that the difference in 
preference intensity between x and y is twice as large as 
the difference between y and z. Ratio-scale utility func- 
tions are quite rare; they show up, for example, in pros- 
pect theory (see chapter 8.3 by Gléckner, this handbook) 
and in experienced-utility measures (see section 3.1). 

Interval differences are particularly important for 
analyzing preferences over lotteries under the expected 
utility model. Lotteries are ordered tuples of outcome- 
probability pairs that are mutually exclusive and jointly 
exhaustive. For example, [x, p; y, g; Z] is the lottery that 
gives outcome x with probability p, y with probability q, 
and z with probability 1-p-q. Under the expected util- 
ity model, the utility of a lottery is the sum of the utili- 
ties of its outcomes, weighted by their probabilities (von 
Neumann & Morgenstern, 1947): 

U[Xy, Pri Xo Pri» + + i Xn» Pol) =>) p,x u(x). (18) 

i=1 

In the example, then, u([X, p; y, q; Z]) = px u(x) +qxu(y) + 
(1-p-q) xu(z). This is a very powerful tool for analyz- 
ing preferences over lotteries, making them comparable 
to preferences over certain outcomes. But it requires an 
interval utility function. For example, a comparison of 
lottery [x,p;y,q;z] to certain outcome y depends not only 
on whether x>y>z but also on whether the difference 
in preference intensity between x and jy, weighted by p, is 
larger than the difference between y and z, weighted by 
1-p-q. Such information cannot be provided by an ordi- 
nal scale but requires an interval-scale interpretation of u. 

Interval-scale utilities are represented by sets of func- 
tions, where each member u of the set is a positive linear 
transformation of another member w’: u(x) = axu'(x) +b, 
where a > 0. An interval-scale representation of a prefer- 
ence relation > is thus a proper subset of an ordinal- 
scale representation of >. For such a representation 
to exist, and to be unique up to positive linear trans- 
formation, however, > must satisfy certain properties. 
Different approaches to these representations include 
Ramsey (1928), von Neumann and Morgenstern (1947), 


Savage (1954/1972), Jeffrey (1965/1983), and Fishburn 
(1970). There are substantial differences between these 
approaches and their respective assumptions, but the 
rationality of at least some of the required preference 
properties in each of these approaches is controversial. 
Thus, in addition to the above-discussed limitations of 
ordinal utility representations, the interval scale further 
restricts which rational preferences utility functions can 
represent. For more detail, see chapter 8.2 by Peterson 
on decision theory (in this handbook). 

Interval utility scales represent differences in intensities 
of preference. This allows describing two abstract proper- 
ties: the relation between utility and quantities of goods 
as well as the relation between utility and uncertainty. 

When evaluating a quantifiable good or service, the 
marginal utility is the value added by the last increment 
of that good or service. For example, if the value of x 
sheep is u(x) = 15 and that of x + 1 sheep is u(x + 1) = 17, 
then the marginal utility of adding one sheep to the x 
previous ones is 2. The so-called law of diminishing mar- 
ginal utility (also known as a “Gossen’s first law”; Gossen, 
1854/1983) states that marginal utilities decrease, ceteris 
paribus, as additional amounts of a good or service are 
added to available resources. One striking implication of 
this principle is that any good, however much valued 
in its first unit, will eventually be valued less than even 
the first unit of a good that is valued rather little. This 
insight solves the paradox of value (famously presented 
by Smith, 1776/1904): water commands a lower mar- 
ket price than diamonds, despite its being on the whole 
more useful, in terms of survival, than diamonds. Why? 
Because most people are well supplied with water and 
thus value an additional unit very little. By contrast, dia- 
monds are scarce, and many people value the first unit 
of diamonds higher than the nth unit of water. 

When evaluating lotteries, risk attitudes express evalu- 
ations of the respective uncertainties of the alternatives. 
For example, a lottery [$100, 0.5; $0] has an expected 
value of $50; those being indifferent between the lottery 
and $50 for certain are called risk neutral, those prefer- 
ring the certain outcome risk averse, and those preferring 
the lottery risk loving. It is widely agreed that moderate 
risk aversion is perfectly rational at least in some cases— 
for example, when someone with few resources prefers 
a certain gain of $100,000 to an even gamble between 
receiving $200,000 or nothing (Arrow, 1971). 

The standard von Neumann-—Morgenstern (vNM) 
expected utility model allows for the representation 
of such risk attitudes but only by equating them with 
changing marginal utilities. By definition (18), the util- 
ity of lottery [$100, 0.5; $0] is the probability-weighted 
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sum of its outcomes; it can differ from the utility of $50 
for certain only if the utilities of outcomes are not linear 
with the monetary amounts. Consequently, general risk 
aversion in the vVNM model is identical to diminishing 
marginal utility. 

This is conceptually and empirically problematic. Con- 
ceptually, evaluations of quantities and of uncertainty 
seem distinct. For example, Bengt Hansson (1988) imag- 
ines a professional gambler who turns down an offer to 
trade a single copy of a book he likes for an even-chance 
gamble between receiving no copy of the book and three 
copies. According to the vVNM model, the gambler must 
be risk averse. Yet the gambler might reject such an analy- 
sis: being a professional gambler, he has habituated him- 
self to being risk neutral. The reason he turns down the 
gamble, he says, is simply that the second and third copies 
are of almost no worth to him. Thus, he exhibits dimin- 
ishing marginal utility for additional books but no risk 
aversion. 

Furthermore, the vNM account cannot represent vari- 
ous empirically observed choice patterns, whose inter- 
pretation in terms of risk preferences seems normatively 
legitimate. For example, a person rejecting a small-stakes 
gamble (e.g., [$100, 0.5; -$100]) can be represented in 
the vNM model only with a diminishing utility func- 
tion for money, which would entail that the same per- 
son would reject very reasonable large-stakes gambles 
(e.g., [$10'°, 0.5; -$100]) (Rabin, 2000). Other examples 
concern choice patterns exhibited in the famous Allais 
and Ellsberg paradoxes, which cannot be represented 
by vNM utilities but are often considered expressions of 
rational attitudes toward risks (see chapter 8.2 by Peter- 
son, this handbook). 

The root of these problems seems to be the identifica- 
tion of risk attitudes with nonlinear evaluation of quan- 
tity of goods in the vVNM model.” Various solutions to 
these problems thus propose to separate the two. One 
approach is to replace (18) with a risk-weighted expected 
utility model (Buchak, 2013; Quiggin, 1982), where the 
agent’s risk function r weighs the probabilities p in a 
nonlinear fashion: 


n 


sith Pall = rue). 


i=1 


U([X1, Pr} X2» Pri - (19) 
In particular, risk aversion would be represented by a 
convex risk function (e.g., r(p) = p’). This representation, 
however, syntactically separates risk attitudes from eval- 
uations. Critics have argued that risk attitudes are a kind 
of desire and therefore propose to model them in the Jef- 
frey framework as desirabilities over chance propositions 
instead (Stefansson & Bradley, 2019). 
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3. What Do Preferences and Utilities Represent? 


The above two sections concerned formal properties of 
representational tools. Although these properties of course 
impose constraints, it turns out that these constraints 
are not sufficient to uniquely determine what prefer- 
ences and utilities represent. Instead, multiple coherent 
positions have been developed, and lively controversies 
in philosophy and the social sciences continue between 
them. I will distinguish three broad categories: (1) those 
that consider utilities as representing mental states— 
like pleasure or happiness—and consider preferences as 
derived from utilities, (2) those that consider preferences 
as representing mental states—like comparative likings 
or judgments—and derive utilities from preferences, and 
(3) those that consider preferences as representing choice 
patterns and derive utilities from preferences. 


3.1 Utility as Representing Mental States 

Although earlier authors had proposed the maximiza- 
tion of happiness as a moral principle, is it only with 
Bentham (1789/1988) that happiness is explicitly identi- 
fied as the presence of pleasure and the absence of pain 
that people feel as the consequence of action. Bentham 
argued that happiness can be measured in its intensity, 
duration, uncertainty, and remoteness and proposed to 
call this aggregate measure utility. 

Bentham proposed utility in this sense only as a 
normative criterion of what should be considered mor- 
ally good. Early marginalist economists like Gossen 
(1854/1983) and Jevons (1871) instead aimed to explain 
social phenomena, based on the motives of (representa- 
tive) human individuals, and they used the utility con- 
cept for that purpose.’ In Jevons’s (1871) words, they 
“attempted to treat Economy as a Calculus of Pleasure 
and Pain.” Furthermore, they created the utility func- 
tion as a cardinal representation of mental states. In this 
framework, the notion of preference, to the extent that it 
was used at all, was merely derived from hedonistic utility. 

While this perspective has been largely replaced in 
20th-century social science, it has recently seen some- 
thing of a revival in Daniel Kahneman’s theory of 
experienced utility. Kahneman assumes that people at 
every moment experience what he terms instant utility— 
meaning pleasure and/or pain. This utility has quantity 
and valence, with a neutral point on the boundary between 
desirable and undesirable, between pleasure and pain, and 
can be measured on a ratio scale. By integrating instant 
utility over a period, the total utility for that period can 
be computed. Given that on this approach, utility can be 
measured independently, utility maximization becomes 
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an empirical, testable hypothesis. To the extent that util- 
ity maximization is seen as a criterion of rationality, the 
experienced-utility approach claims to offer an empirical 
test for the rationality of individuals in particular domains 
of behavior (Kahneman, Wakker, & Sarin, 1997). 


3.2 Preferences as Representing Mental States 

In the social sciences, the preference concept became 
important for explanatory and predictive purposes 
with Irving Fisher’s (1892/1961) and Vilfredo Pareto’s 
(1906/2014) methodological criticisms of hedonic cardi- 
nal utility. Pareto argued that because an accurate mea- 
surement procedure for cardinal hedonic utility was not 
available, social scientists should constrain themselves 
to merely ordinal comparisons (Bruni & Guala, 2001). 
This argument turned preference into a fundamental 
notion of the social sciences, replacing (hedonic) utility. 

Economists in the 1930s (Hicks & Allen, 1934) radical- 
ized Pareto’s idea and argued that cardinal utility should 
be excluded in order to purge economics of psycho- 
logical hedonism. However, their concept of preference 
retained psychological content: people were assumed to 
act purposefully and therefore to have preferences that 
really constitute mental evaluations, rather than being 
ex post rationalizations of behavior (Lewin, 1996). 

Based on (18) and stated preferences over lotteries with 
known probabilities, von Neumann and Morgenstern 
devised a measurement of interval utility. Later, Savage 
(1954/1972) and Jeffrey (1965/1983) devised simulta- 
neous measurements of utilities and probabilities from 
stated preferences. These new concepts of utility, how- 
ever, were very different from the older hedonic concept: 
here the preference concept is basic and the cardinal util- 
ity function merely derived." 

Because utility is derived from preferences using repre- 
sentation theorems, the assumptions necessary for these 
theorems are built into the utility representation. Utility 
representations of this kind thus already assume rational- 
ity properties like transitivity, independence, and conti- 
nuity. It would therefore be pointless to try to test whether 
such utility representations meet these rationality criteria 
(cf. Davidson’s arguments discussed in section 2.3). 


3.3. Preferences as Representing Choice Patterns 

In economics, the revealed-preference approach defined 
preference in terms of choice. Historically, this approach 
developed out of the pursuit of behavioristic foundations 
for economic theories—that is, the attempt to eliminate 
the mental-preference interpretation altogether. Specifi- 
cally, the downward slope of the demand function—one 
of the central features of microeconomics—could be 


derived only from properties of marginal utilities, yet 
these properties were difficult to measure with the men- 
talistic Allan—Hicks utility concept. Consequently, Paul 
Samuelson in 1938 proposed to “start anew... drop- 
ping off the last vestiges of the utility analysis” (Samuel- 
son, 1938, pp. 61-62). 

The basic idea was to assume that choice satisfied cer- 
tain rationality postulates and then define a revealed pref- 
erence relation x =z y between two goods bundles x and y 
as the choice of x in conditions where either x or y could 
have been chosen.° In other words, if an individual at 
current prices can afford both purchasing x and purchas- 
ing y, given her budget, and she chooses to purchase x, 
then she is said to revealed-prefer x to y. Based on the above 
definitions (1) and (2), revealed indifference and revealed 
strict preferences can be derived from +x. Furthermore, by 
determining revealed preferences over relevant lotteries 
and applying one of the expected utility frameworks dis- 
cussed in section 2.2, these preferences can be represented 
on an interval utility scale. Note, however, that in this 
case, the utility function does not represent any mental 
content at all but only recorded choice patterns. 

Although this approach was highly influential at the 
time, not all economists followed Samuelson in this 
radical proposal—so that today, the mentalist and the 
revealed-preference interpretation of preferences are often 
both presented in textbooks (e.g., Mas-Colell et al., 1995). 
Indeed, it might be the case that Samuelson himself later 
changed his mind, shifting from the definition of preference 
in terms of choice to a measurement of preference based on 
choice data (Hands, 2014). Nevertheless, there are many 
economists today who defend the behaviorist interpreta- 
tion, for example, by denying that preferences represent 
the causes of choice (Binmore, 2009) and by insisting on 
“mindless economics” (Gul & Pesendorfer, 2008). Con- 
currently, there is an ongoing discussion among philos- 
ophers whether the current concept of preference used 
by economists is a separate theoretical concept—defined 
against the common use as capturing choice patterns—or 
whether it indeed is based on the mental, “folk-theoretic” 
notion (Maki, 2000; Ross, 2014). 

Critics of the behaviorist interpretation have pointed 
out that the common “folk” explanations of action 
require both a motivational component (represented 
by, e.g., preferences or utility) and an epistemic com- 
ponent (i.e., beliefs represented by, e.g., probabilities). 
The revealed-preference account neglects this epistemic 
component and therefore cannot provide folk explana- 
tions (Hausman, 2012; List & Dietrich, 2016; Sen, 1993). 
While the defenders of revealed preference might con- 
cede as much (see above), Hausman further argues that 
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the folk-psychological account already provides a lexical 
definition of preferences and that it is counterproduc- 
tive to craft a stipulative definition of preferences that is 
supposed to replace this folk notion in economics. 

However, defenders of revealed preference further 
argue that it is necessary to distinguish between those 
agents who indeed have preferences as states of minds 
(e.g., humans and maybe higher animals) and those 
agents who do not (e.g., machines, plants, or institu- 
tions). The former category might choose on the basis 
of their mentalistically interpreted preferences, and 
hence the above-discussed effort can aim at eliciting the 
preferences on which their choices are based. The lat- 
ter category, despite their lack of states of mind, might 
nevertheless exhibit behavior that can be interpreted as 
rational choice. In those cases, one can only speak of 
preferences reconstructed from choice, without claiming 
that these preferences describe mental states at all (Gul 
& Pesendorfer, 2008; Ross, 2014). 


4. Preferences Combination 


A preference relation = might be constructed from a 
vector of preferences (+1,..., =n). Such a relationship 
has a number of different interpretations. For example, 
on an intrapersonal interpretation, = might be the total 
preference of an individual—that is, an overall compari- 
son of two relata, taking all relevant considerations into 
account. The items of (+1,..., =n) might be the par- 
tial preferences of this individual—for example, desir- 
able characteristics of economic goods (Lancaster, 1966) 
or different reasons that one may have to prefer one 
of the options to another (Dietrich & List, 2013; Pettit, 
1991). For example, =; might represent evaluations of the 
respective health benefits of different foodstuffs, while +2 
might represent the comparative sustainability of their 
production. 

An interpersonal interpretation, in contrast, would 
interpret (=1,...,%) as the collection of total prefer- 
ences of individuals 1,..., m in a group and = as the 
group preference. For example, the managing board of 
a company might consist of m members, each of whom 
has a preference for a certain course of action. The final 
decision of which action to take might be interpreted as 
the expression of the group preference >. 

Some authors have argued that for intrapersonal cases, 
total preferences are always the result of an aggregation of 
partial preferences (e.g., Hausman, 2012). These authors 
assume that a total preference relation is uniquely deter- 
mined by the partial preference relations through a 
process of aggregation. 
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Other authors reject the idea that total preferences 
are uniquely derivable from partial preferences. Instead, 
they claim that total preferences are constructed at the 
moment of elicitation and thus influenced by contexts 
and framings of the elicitation procedure that are not 
encoded in preexisting partial preferences (Payne, Bett- 
man, & Johnson, 1993). Total preferences seem to be 
influenced by direct affective responses that are indepen- 
dent of cognitive processes (Zajonc, 1980). For instance, 
food preferences seem to be partly determined by habit- 
uation and are therefore difficult to explain as the out- 
come of a process exclusively based on well-behaved 
partial preferences. According to this view, partial prefer- 
ences are in many cases ex post rationalizations of total 
preferences rather than the basis from which total pref- 
erences are derived. 

If total preferences are the result of an aggregation of 
partial preferences, the question arises what this aggre- 
gation process might be. Two kinds of approaches can be 
distinguished. First, one could seek to develop intraper- 
sonal aggregation by following models of social choice 
that describe interpersonal aggregation of preferences 
(Arrow, 1963; Sen, 1970). However, such models do not 
allow for trade-offs between partial preferences. That is, 
if a particular x is much better than a y in one dimen- 
sion k, the preference y > x might nevertheless be possi- 
ble because the strength of y in other dimensions over x 
outweighs dimension k. This approach requires (i) a car- 
dinal measure of preference intensity and (ii) the compa- 
rability of these preference intensities across dimensions 
(Keeney & Raiffa, 1993). 


5. Preference Change 


Preferences sometimes change. This poses difficult pre- 
dictive and explanatory challenges for social scientists, 
many of which remain currently unsolved. In this sec- 
tion, I will instead focus on the question of what might 
constitute rational preference change, discussing three 
approaches: temporal-discounting models, consistency- 
restoring preference revision models, and doxastic pref- 
erence change (for a broader overview, see Griine-Yanoff 
& Hansson, 2009). 


5.1. Temporal-Discounting Models 

Sometimes preferences change because the temporal dis- 
tance between the time of evaluation and the realization 
of the evaluated event changes (see chapter 10.4 by Raub 
on rational choice, this handbook). For example, many 
people consider a tedious task to be performed in three 
months’ time less bad than the same task to be performed 
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now, even if all background conditions are equal. As this 
is an intensity change, preferences must be represented 
on an interval scale. The standard model then represents 
the utility u(x, of an alternative x at time t as equal to 
the time-independent utility of x (which one might think 
of as the utility of x if it were realized now), discounted 
by the delay: 


u(x,t) = u(x)/(1+n', (20) 


where r is the discount rate. This is the exponential dis- 
counting model, proposed by Samuelson (1937), which still 
dominates economic analysis. It is a fact that people at 
least sometimes do not discount in agreement with the 
exponential model. Various nonexponential models have 
been proposed, of which the hyperbolic discounting model 
is the most prominent. 

While it is obvious that the behavior of many people 
can only be described by assuming some kind of time- 
discounting, the question remains whether such dis- 
counting is rational and justifiable. At least two questions 
need to be distinguished—first, whether discounting util- 
ity of a prospect merely for its distance in time is ever 
justified, particularly whether steep discount rates can 
be justified—irrespective of what the particular form of 
the discounting function is. Critics argue that one should 
want one’s life, as a whole, to go as well as possible and 
that counting some parts of life more than others inter- 
feres with this goal (Pigou, 1920; Ramsey, 1928; Rawls, 
1971). According to this view, it is irrational to prefer a 
smaller immediate good to a greater future good, because 
now and later are equal parts of one life. Choosing the 
smaller good or the greater bad makes one’s life, as a 
whole, turn out worse: “Rationality requires an impartial 
concern for all parts of our life. The mere difference of 
location in time, of something’s being earlier or later, is 
not a rational ground for having more or less regard for 
it” (Rawls, 1971, p. 293). Critics of temporal discounting 
often attribute apparent departures from temporal neu- 
trality to a cognitive illusion, which causes people to see 
future pleasures or pains in some diminished form. 

Against the temporal neutrality of preferences, some 
have argued that there is no enduring, irreducible entity 
over time to whom all future utility can be ascribed; they 
deny that all parts of one’s future are equally parts of 
oneself (Parfit, 1984). They argue, instead, that a person 
is a succession of overlapping selves related to varying 
degrees by memories, physical continuities, similarities 
of character and interests, and so on. On this view, it 
may be just as rational to discount one’s “own” future 
preferences as to discount the preferences of another, 
distinct individual, because the divisions between the 
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stages of one’s life may be as “deep” as the distinctions 
between individuals. 

The second question concerns the particular form of 
discounting. One of the central properties of Samuel- 
son’s exponential model is that it is order-preserving. That 
is, an individual who prefers (y,t + i) to (x,t) at some time 
previous to t will also prefer (y,t+i) to (x,#) at time ft. 
The only difference between these two evaluations is the 
increase in utility for both (y,t+i) and (x,t). The expo- 
nential model ensures that these intensities increase 
proportionally, so that the preference ranking of options 
never changes. 

Many people experience temporal preference changes 
that contradict this order-preserving property: they start 
out preferring (y,t+i) to (x,t) at some earlier time— 
perhaps because y, when compared to x at the same time, 
is so much better. But as they approach ft, and hence the 
realization of x, they experience a preference reversal: near 
to and at t, they suddenly prefer (x,#) to (y,t+i). Fora 
simple example, consider a person who prefers one apple 
today to two apples tomorrow but (today) prefers two 
apples in 51 days to one apple in 50 days. Although this 
is a plausible preference pattern, it is incompatible with 
the exponential model. It can, however, be accounted 
for in a model with a declining discount rate. Pioneered 
by Ainslie (1992), psychologists and behavioral econo- 
mists have therefore proposed to replace Samuelson’s 
exponential-discounting model with a model of hyper- 
bolic discounting. The hyperbolic model discounts the 
future consumption with a parameter inversely propor- 
tional to the delay of consumption and hence can repre- 
sent preference reversals like the above (for an overview, 
see Griine-Yanoff, 2015). 

Choosing according to hyperbolic-discounted util- 
ity leads to periodic abandonment of previous consump- 
tion plans in favor of new ones, which would be in turn 
abandoned in the next period. Such patterns of time- 
inconsistent choice are often related to failures of self- 
control, and hyperbolic discounting is therefore often seen 
as an expression of irrational preferences (Strotz, 1956). 

However, some authors have argued that hyperboli- 
cally discounted preferences are adaptive for environ- 
ments in which the uncertainty of obtaining a distant 
reward increases nonlinearly with the temporal distance 
(Sozou, 1998). In the light of these results, hyperbolic 
discounting might be rational at least in some contexts. 
Against this, others have argued that it might be possi- 
ble to separate these considerations of uncertainty from 
pure time preferences (Andreoni & Sprenger, 2012) and 
that nonexponential discounting of pure time prefer- 
ences remains a sign of irrationality. 
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5.2 Consistency-Restoring Preference Revision Models 
If an agent forms a specific preference as the result of 
some experience, further changes in her overall pref- 
erence state are often necessary to regain consistency. 
Using tools from belief revision (see chapter 5.2 by Rott 
on belief revision, this handbook), preference change has 
been modeled as an adjustment to such inputs (Griine- 
Yanoff, 2013; S. O. Hansson, 1995; Liu, 2011). Changes 
in preference are triggered by inputs that are represented 
by sentences expressing new preference patterns. 

For example, if a subject grows tired of her previous 
favorite brand of mustard, x, and starts to like brand z 
better, then this will be represented by a change with the 
sentence “z is better than x,” in formal language: z > x, as 
an input. However, a change in which the previous pref- 
erence x > z is substituted by the new preference z> x can 
happen in different ways. For instance, there may be a 
third brand, y, that was previously placed between x and 
z in the preference ordering. The instruction to make the 
new preference relation satisfy z > x does not tell us where 
y should be placed in the new ordering. The new order- 
ing may, for instance, be either z>x>y or z>y>x. One 
way to deal with this is to include additional information 
in the input, for instance, specifying which element(s) of 
the alternative-set should be moved while the others keep 
their previous positions. In my example, if only z is going 
to be moved, then the outcome should satisfy z>x>y. 
These and other considerations make it necessary to mod- 
ify the standard model of belief change in order to accom- 
modate the subject matter of preferences. 


5.3 Doxastic Preference Change 
Preferences also change as a consequence of belief 
changes. Two kinds of beliefs are especially important 
here. The first is the belief that the presence of state x 
would make a desired state y more likely—a rise in prob- 
ability of y given x produces a rise in the desirability 
of x, and vice versa. The second kind of belief relevant 
for doxastic preference change concerns prospects that 
influence the preference for other prospects without 
being probabilistically related. For example, one’s pref- 
erence for winning a trip to Florida in the lottery will 
crucially depend on one’s belief about the weather there 
during the specified travel time, even though these two 
prospects are probabilistically unrelated. More generally, 
if x a y is preferred to x a -y, with x and y probabilisti- 
cally not correlated, then a rise in the probability of x 
will result in a rise in the desirability of y (even if it does 
not affect the probability of y), and vice versa. 

Jeffrey (1977) provided a simple model of preference 
change as the consequence of an agent’s coming to believe 
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a proposition A to be true. His model incorporates both 
kinds of belief relevant for doxastic preference change. 
The basic idea is that (u,P) represents the unconditional 
preference > if P is the probability distribution based on 
the agent’s actual information. The conditional preference 
ordering =a, in contrast, is represented by the tuple (u, Pa), 
where P, is the probability distribution based on the coun- 
terfactual scenario that the agent accepts proposition A as 
true. That is, the agent imagines that if he changed his 
whole belief system from P to P,, then he would have the 
preference relation =, as represented by (u,P4) (for more 
discussion of conditional preferences, see Bradley, 2005; 
Joyce, 1999; Luce & Krantz, 1971). 


Notes 


1. The study of these properties can be traced back to Book II 
of Aristotle’s Topics. Since the early 20th century, several phi- 
losophers have studied the structure of preferences with logical 
tools. In 1957 and 1963, respectively, S6ren Halldén and Georg 
Henrik von Wright proposed the first complete systems of pref- 
erence logic (Halldén, 1957; von Wright, 1963). 


2. This identification has a long tradition, going back to Daniel 
Bernoulli’s solution to the St. Petersburg paradox (Bernoulli, 
1738/1954). 


3. Jevons discussed the effect of character, gender, and race 
on the utility function and avoided dealing with the implied 
heterogeneity of utility functions by invoking such a represen- 
tative individual (White, 1994). 


4. Defenders of the Benthamite conception call this new VNM 
representation decision utility, to distinguish it from their expe- 
rienced utility. 


5. The two main assumptions are the weak axiom of revealed 
preferences (WARP) and the strong axiom of revealed preferences 
(SARP). WARP says that if x is chosen when y is available 
(in some alternative-set A), then there must not be another 
alternative-set A’ containing both alternatives for which y is 
chosen and x is not. SARP says that if from a set A, of alterna- 
tives, x is chosen when y and z are available, and if in some 
other set of alternatives, Az, y is chosen while z is available, 
then there can be no set of alternatives containing alternatives 
x and z for which z is chosen and x is not. (SARP says this for 
chains of any length.) 
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Summary 


What is it rational to do in a choice situation, given the 
limited and sometimes unreliable information available 
to an agent? The dominant view among decision theorists 
is that rational agents act in accordance with the principle 
of maximizing expected utility. This chapter surveys the 
debate on the expected utility principle and discusses the 
most significant objections raised in the literature. 


1. Right versus Rational Decisions 


Decision theory is a theory of practical rationality that 
seeks to answer the following question: what is it ratio- 
nal to do in a choice situation, given the limited and 
sometimes unreliable information available to an agent? 
Decision theorists distinguish between right decisions 
and rational ones. If you play the National Lottery and 
win, your decision was right (because the outcome was 
optimal), but if the odds of winning were as long as they 
usually are (and the act of playing the lottery did not 
in itself give you great pleasure), the decision was irra- 
tional. In light of the information available to you, you 
had more reason to abstain from playing than for doing 
so: the value of hitting the jackpot did not outweigh the 
low probability of winning. The task decision theorists set 
themselves is to articulate and defend this line of thought. 
How, exactly, should information about an act’s possible 
outcomes guide our decisions? This chapter surveys some 
of the most influential answers and discusses a number of 
significant objections raised in the literature. 

A fundamental assumption accepted by more or less 
all decision theorists is that the notion of rationality 
described by decision theory is a means—ends notion 
in which the rationality of a decision depends on the 
agent’s beliefs and desires (see chapter 2.1 by Broome, 
this handbook). Decision theorists have little to say 
about what beliefs and desires are, but they assume 
that beliefs and desires can vary in strength. It is widely 
believed that the agent’s degree of belief in something 


can be represented by a subjective probability function 
(see section 4 of this chapter and chapter 4.1 by Hajek 
& Staffel, this handbook) and that desires can be rep- 
resented by a utility function. The stronger you believe 
something and the more you desire it, the higher is your 
subjective probability and utility. 


2. The Principle of Maximizing Expected Utility 


Pascal (1670/1967), Nicolaus Bernoulli (in a letter from 
1713), and Cramer (1728/1975) propose that an act is 
rational if and only if its expected utility is at least as high 
as that of every alternative act.' This has been the domi- 
nant view among decision theorists ever since, but there 
is significant disagreement about how the expected utility 
principle should be interpreted and justified. The expected 


utility of an act with n possible outcomes xj, .. . , Xn is 


DP H(%), (1) 
i=1 
where p; is the probability of outcome x; and u(x) the 
utility of x; (see chapter 8.1 by Griine-Yanoff, this hand- 
book). If the number of possible outcomes x is infinite, 
the expected utility is J vexL P(X) + u(x) dx. 


3. The Law of Large Numbers 


A possible reason for accepting the principle of maximiz- 
ing expected utility (or expected value more generally) 
is that agents who do so will almost certainly be better 
off in the long run. This insight is formalized in a math- 
ematical theorem known as the law of large numbers:’ if 
a random experiment for which the expected value is Ey 
is repeated n times, then the probability that the average 
value X of the experiment differs from E, by more than 
some small number ¢ converges to 0 as the number of 
experiments n goes to infinity, and this holds true for 
every positive ¢>0O no matter how small: 


lim p(X - E,|>6)=0. (2) 
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This mathematical insight might be of some impor- 
tance for agents who face similar choice situations a large 
number of times. However, many decisions are unique. 
Claims about what would happen if one were to make deci- 
sions that one will not actually make therefore seem irrel- 
evant. Would it, for instance, be rational for you (given 
your interests and financial circumstances) to study at 
university X rather than university Y? Should you buy the 
house offered for sale for $400,000, or should you look for 
a cheaper house? Would it be rational to quit your job to 
sail around the world? When answering these questions, 
it seems irrelevant what would happen if you were to face 
similar choices a million times, because you know that you 
will only face each choice a small number of times. Keynes 
(1923/2000) expressed this insight in its most brutal form: 
“In the long run we are all dead” (p. 80). 


4. Axiomatic Analyses of the Expected Utility Principle 


4.1. Von Neumann and Morgenstern 

Several authors have developed axiomatic analyses of 
the expected utility principle, which are sometimes inter- 
preted as normative arguments for accepting the princi- 
ple. See, for example, Ramsey (1926/1931), von Neumann 
and Morgenstern (1947), Savage (1954/1972), and Jeffrey 
(1983).? None of these axiomatizations entail that rational 
agents will make their decisions by consciously multiplying 
probabilities and utilities. The axioms merely entail that 
agents forming preferences in accordance with the axioms 
can be described as if they were maximizing expected util- 
ity. As Blackburn (1998) puts it, these representation and 
uniqueness theorems provide us with a “grid for impos- 
ing interpretation: a mathematical structure, designed to 
render processes of deliberation mathematically tractable, 
whatever those processes are” (p. 135). 

Von Neumann and Morgenstern focus on decision 
problems in which probabilities can be defined exog- 
enously, meaning that the agent somehow knows the 
probability of every event relevant to her decision. In 
this section, we will use a slightly different notation 
than that used in chapter 8.1 by Grtine-Yanoff (this 
handbook). Recall that von Neumann and Morgenstern 
conceive of acts as lotteries: xpy is the lottery that yields 
xX with probability p and y with probability 1—p (and 
this holds regardless of whether x and y are basic prizes 
or new lotteries). The possible outcomes x or y are either 
basic prizes (such as money or a ticket to the opera) 
or lotteries over basic prizes. The relation > is a binary 
preference relation over the set of all possible lotteries L. 
Von Neumann and Morgenstern propose that agents are 
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rationally required to form preferences over lotteries that 
meet the following structural requirements, for all lotter- 
ies and basic prizes x,y,z and all probabilities p and q: 


vNM 1 (Weak Order) = is a complete, reflexive, 


and transitive relation on L. 
vNM 2 (Independence) xy if and only if xpz > ypz. 


vNM 3 (Continuity) If x=y ez, then there exist 
some p and g such that 


XPZ Fy = xqzZ. 


The independence axiom is the most controversial of 
the three axioms. In table 8.2.1, it seems reasonable to 
prefer lottery x to y, because this will give you $1 mil- 
lion for sure. However, the independence axiom entails 
that if you prefer x to y, then you must also prefer xpz to 
ypz. This is counterintuitive. It seems perfectly reasonable 
to prefer ypz to xpz, because there is no certain outcome 
here. The .10 probability of winning $5 million instead of 
$0 might very well be considered more attractive than a 
slightly larger probability (.11) of winning a much smaller 
amount ($1 million). For discussions of the ordering and 
continuity axioms, see chapter 8.1 by Griine-Yanoff (this 
handbook). 

Von Neumann and Morgenstern’s theorem states 
that whenever vNM 1-3 are satisfied, there exists a real- 
valued function u(x) that takes a lottery in L as its argu- 
ment and returns a real number between 0 and 1, which 
has the following properties: 


(1) x= yif and only if u(x) = u(y). 

(2) u(xpy) = p-u(x) + (1-p)-ugy). 

(3) For every other function u’ satisfying (1) and (2), there 
exist some constants c>0O and d such that uw’ = cu+d. 


Properties (1) and (2) constitute the representation part: 
the agent assigns higher utility to outcomes she judges to 
be preferable, and the utility of a lottery equals the expected 
utility of its outcomes. Property (3) is the uniqueness part: 
any other function u' that satisfies the first two properties 


Table 8.2.1 
The independence axiom 


Ticket 1 Ticket 2-11 
Lottery x $1 million $1 million 
Lottery y $0 $5 million 
Ticket 
Ticket 1 Ticket 2-11 12-100 
Lottery xpz $1 million $1 million $0 
Lottery ypz $0 $5 million $0 
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is a linear transformation of u, meaning that utility is mea- 
sured on an interval scale (see chapter 8.1). 

A common objection to normative interpretations of 
von Neumann and Morgenstern’s result is that it puts the 
cart before the horse (for an overview, see Peterson, 2008, 
section 2.5). The agent acting in accordance with these 
axioms does not prefer an act because its expected utility 
is optimal. She can be described as if she were acting from 
the expected utility principle, but we have no reason to 
believe that the utility function we ascribe to the agent 
is equivalent to the function that guides her choices or 
that such a function exists. The problem is thus that the 
theory is not action-guiding. The ordering axiom explic- 
itly requires that agents have complete preferences over 
all possible lotteries (acts), including very complex lotter- 
ies (acts), such as the ones mentioned earlier: Would it be 
rational for you to study at university X rather than uni- 
versity Y? Should you buy the house offered for sale for 
$400,000, or should you wait and hope the price drops 
to $350,000? Should you quit your job to sail around the 
world? Agents who have complete preferences over all pos- 
sible lotteries hardly need advice from decision theorists. 

The standard response to this objection is that the axi- 
oms could be action-guiding in an indirect sense. They 
can be treated as “inference rules” that can help noni- 
deal agents revise rationally impermissible preferences. If 
you, say, discover that your preferences are incomplete, or 
intransitive, then the axioms tell you how to overcome this 
deficiency: revise your preferences, in whatever way you 
wish, so they meet the structural requirements imposed by 
the axioms. However, this type of indirect action guidance 
seems somewhat unhelpful. If you are trying to figure out 
if it is rational for you to study at university X or university 
Y or whether to offer $400,000 for a house you like, and 
the decision theorist is merely able to tell you that “any- 
thing goes” as long as the axioms are fulfilled, then decision 
theory will not be of much help to you. 


4.2 Savage 

The axiomatizations proposed by Ramsey (1926/1931), 
Savage (1954/1972), and Jeffrey (1983) differ from von 
Neumann and Morgenstern’s in that probabilities are 
derived from the axioms themselves. In these axiomat- 
izations, the probability function is not an exogenously 
defined function. Furthermore, since the axioms regu- 
late the agent’s preferences, we can interpret probabili- 
ties derived from these axioms as subjective probabilities, 
or credences (see chapter 4.1 by Hajek & Staffel, this 
handbook). To say that the probability is .90 that it will 
rain tomorrow means that you believe to a fairly strong 
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degree that it will rain tomorrow: your credence makes 
you willing to pay something worth .90 units of utility 
for a bet in which you win 1 unit if it rains tomorrow 
and nothing otherwise. 

Savage conceives of acts as functions from a set of 
possible states of the world to outcomes. The act of not 
bringing the umbrella to the football game is a function 
in which the two possible states are “rain” and “no rain,” 
and the possible outcomes are “the agent gets wet” and 
“the agent does not get wet,” depending on which state 
turns out to be the true state of the world. 

Some of the axioms proposed by Savage resemble 
those proposed by von Neumann and Morgenstern. This 
includes the assumption that the agent’s preferences 
satisfy the weak ordering axiom, which raises the same 
issue about putting the cart before the horse discussed 
earlier. Another axiom borrowed from von Neumann- 
Morgenstern (and, ultimately, from Ramsey, 1926/1931) 
is the so-called sure-thing principle. This is Savage’s name 
for a much-discussed version of the independence 
axiom. In Savage’s terminology, two acts f and g agree 
with each other in a set S of states if and only if f(s) = g(s) 
for all s ES. In this terminology, the sure-thing principle 
can be formulated as follows: 


If fand g, and f' and g’, are such that 


1. in =S, the complement of S, f agrees with g, and f' 
agrees with g’, 

2. in S, fagrees with f', and g agrees with g’, and 

3. fF FS, 

then f' +g". 


Table 8.2.2 is helpful for highlighting the similarities with 
von Neumann and Morgenstern’s independence axiom. 
Imagine that you are offered a choice among four lotteries, 
each of which has 100 tickets. It seems rationally permis- 
sible to prefer fto g but g’ to f’ (for the reason mentioned 
in the discussion of the independence axiom), but the 
sure-thing principle controversially entails that if you pre- 
fer fto g, then you must prefer f' to g’. 


Table 8.2.2 
The sure-thing principle 


S =S 
Ticket no. 1 Ticket nos. 2-11 —‘ Ticket nos. 12-100 
Actf $1 million $1 million $1 million 
Actg $0 $5 million $1 million 
Act f' $1 million $1 million $0 
Actg’ $0 $5 million $0 


476 


One of the axioms in Savage’s theory, which does not 
figure in von Neumann and Morgenstern’s (in which 
the probability function is defined exogenously), is 
designed to test whether the agent considers two states 
to be equally probable. Savage’s own formulation of this 
axiom is quite complex, but the following explanation 
offered by Broome (1990) is more helpful: 


Savage .. . needs to test whether two events, say E and F, are 
equally probable. He does this by taking a pair of outcomes, 
say A and B, that are known not to be indifferent. He forms 
a gamble (A, E; B, F) in which A comes about in event E 
and B in F. And he forms the opposite gamble (B, E; A, F). 
The events are equally probable if and only if these gambles 
are indifferent. Savage’s Postulate 4 says that this test will 
deliver the same answer whatever pair of non-indifferent 
outcomes A and B are used. (p. 483) 


Broome notes that a problem with this axiom is that 
it seems to assume what it is supposed to show: the 
assumption that the agent will be indifferent between 
the gambles only if they are judged to be equally prob- 
able presupposes that probabilities and utilities are aggre- 
gated in accordance with the expected utility principle. 
Savage’s uniqueness and representation theorem is 
similar to von Neumann and Morgenstern’s, except that 
the probability function is derived from the axioms. The 
theorem guarantees the existence of a utility function that 
is unique up to positive linear transformations and a prob- 
ability function such that one act is preferred to another 
just in case its expected utility exceeds that of the latter. 


4.3 Bolker-Jeffrey 
The theory proposed in Jeffrey (1965) was axiomatized by 
Bolker (1967) and presented in a slightly modified form 
in the second edition of Jeffrey’s book (Jeffrey, 1983). The 
Bolker-Jeffrey theory differs from von Neumann and 
Morgenstern’s and Savage’s in several important respects. 
In the Bolker-Jeffrey theory, it is assumed that the 
agent has preferences over propositions, such as “I have a 
refreshing swim with friends,” rather than lotteries (von 
Neumann & Morgenstern, 1947), or functions from 
states to outcomes (Savage, 1954/1972). Utilities and 
probabilities are also assigned to propositions, mean- 
ing that the Bolker—Jeffrey theory is more economical 
from an ontological point of view. Another philosophi- 
cal advantage is that the use of propositions enables 
the agent to retain her original beliefs about the world 
throughout the deliberative process. There is no need 
for the agent to imagine complex lotteries that require 
the existence of causal processes she does not believe in. 
The agent can continue to believe whatever she believes, 
without adding any new beliefs or modifying old ones. 
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In the Bolker-Jeffrey theory, it is assumed that prefer- 
ences over the Boolean algebra B (with the zero removed; 
contradictions have no place in the preference ordering) 
satisfy the weak ordering axiom, are continuous, and 
satisfy the following two axioms: 


(Averaging) If a, b in B are contraries (i.e., cannot 
both be true), then 
a>bimplies a> (av b)>b, and 
a~bimplies a~ (av b)~b. 
(Impartiality) If a,b,c in B are pairwise contraries, and 


aw~b but not a~c, and (av c)~(bv 0), 
then for every d in B that is contrary to 
aand b, (av d)~(bvd). 


The averaging axiom is somewhat similar to, but 
weaker than, the independence axiom. It requires that 
the agent ranks a disjunction of two propositions some- 
where between the two disjuncts. This may seem reason- 
able. However, a possible reason for ranking a disjunction 
above or below the disjuncts could be that the uncertainty 
introduced by the disjunction sometimes has a positive or 
negative effect on the value of the proposition. Imagine, 
for instance, that you prefer the proposition “My partner 
loves me” to “My partner breaks up with me tomorrow.” 
If so, it does not seem irrational to rank the disjunction 
of these two propositions lower than “My partner breaks 
up with me tomorrow.” Not knowing for sure if your 
relationship is over might be worse than knowing that it 
is. If you learn that your partner will break up with you 
tomorrow, you can adjust your plans for the future and 
move on. 

The impartiality axiom resembles the equiprobabil- 
ity axiom in Savage’s theory. The intuition it seeks to 
capture is similar to that summarized by Broome in the 
quote in section 4.2: to test whether the agent consid- 
ers two propositions to be equally probable, we can ask 
whether the agent is indifferent between a v c and bv c 
whenever she is indifferent between a and D. If the agent, 
for instance, is indifferent between a and b but prefers 
avc over bv c, then she does not consider a and b to 
be equally probable (as long as she maximizes expected 
utility; more about this soon). The impartiality axiom 
states that this holds true for every third proposition c. 

Broome (1990) points out that the impartiality axiom 
assumes what it is supposed to show, just as axiom 4 in 
Savage’s theory. If the agent prefers a to b, then in case 
she maximizes expected utility, the fact that she prefers 
avc over bv c entails that she considers a to be more 
probable than b. However, if she aggregates probabilities 
and utilities in some other way, she has no reason to 
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accept this axiom. It is not entirely satisfactory to justify 
the expected utility principle by appealing to an axiom 
that will be attractive primarily to those who accept the 
expected utility principle. 


4.4 Other Axiomatizations 

A radically different approach to the expected utility 
principle is to start with preferences over certain out- 
comes (basic prizes), as well as beliefs about the world, 
and then generate preferences over risky acts from this 
ex ante information. The ex ante approach steers clear 
of the objection that traditional axiomatizations put 
the cart before the horse. The drawback is that it seems 
somewhat optimistic to think that ordinary agents have 
access to exogenously defined utility and probability 
functions (see Peterson, 2004). 


5. Pragmatic Arguments 


Faced with counterexamples to some of the key axi- 
oms of expected utility theory, decision theorists have 
attempted to offer further warrant for the axioms pro- 
posed by von Neumann and Morgenstern, Savage, and 
others by formulating pragmatic arguments (see, e.g., 
McClennen, 1990). A pragmatic argument is a device 
that seeks to show that anyone who violates a purported 
rationality requirement can be placed in a situation in 
which she has to act contrary to her own preference. 
Donald Davidson, J. C. C. McKinsey, and Patrick Sup- 
pes (1955) propose the following pragmatic argument in 
defense of the transitivity condition, which is key to the 
assumption that rational preferences form a weak order: 


Mt. S. is offered his choice of three jobs by a cynical depart- 
ment head (never mind what department): He can be a full 
professor with a salary of $5,000 (alternative a), an associate 
professor at $5,500 (alternative b) or an assistant professor at 
$6,000 (alternative c). Mr. S. reasons as follows: a> b since 
the advantage in kudos outweighs the small difference in 
salary; b>c for the same reason; c>a since the difference 
in salary is now enough to outweigh a matter of rank... . 
It is clear that the set of Mr. S.’s preferences makes a ratio- 
nal choice impossible, for whichever alternative he chooses 
there will be another alternative which is preferred to it. 
(Davidson et al., 1955, p. 145) 


This argument is sometimes presented in a diachronic 
form, that is, as a series of choices to be made at different 
points in time. Imagine, for instance, that you sign the 
contract for the assistant professorship with a salary of 
$6,000 (alternative c). Right after you do so, the cynical 
department head reminds you that you prefer b to c. He 
offers you to swap if you pay him a small administrative 
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fee, say $.01. Because you prefer b minus the small fee to 
c, you accept his offer. However, right after you sign the 
contract for job b, he offers you to swap b for a if you pay 
a small fee, and once you have paid the fee, he offers you 
to pay him for swapping again, and so on. The cyclical 
structure of your preferences forces you to keep paying 
small amounts for swapping over and over again. After 
100 million swaps, you have lost $1 million but gained 
nothing, which seems to indicate that your preferences 
are irrational. This type of pragmatic argument is known 
as a money-pump argument. 

Schick (1986) claims that a rational agent with cyclical 
preferences will “see which way the wind is blowing... 
[and] seeing what is in store for him, he may well reject 
the offer and thus stop the pump” (p. 117). This is, how- 
ever, a problematic response. Schick’s argument can at 
best cast doubt on the diachronic money pump sketched 
above, but it does not affect the original synchronic ver- 
sion proposed by Davidson et al. (1955). As stressed by 
Gustafsson (2013, p. 462), Davidson et al.’s point is that 
“it is irrational to choose an alternative to which another 
alternative is preferred.” This is irrational even if the agent 
makes only one such choice. There is no need to consider 
diachronic versions of the argument in which the agent 
has to take into account what will happen in the future. 

Another problem with money-pump arguments for 
transitivity is that they do not show that every possible 
violation of transitivity will lead to a sure loss. Prefer- 
ences can violate the transitivity condition in many dif- 
ferent ways, and not every violation forces the agent to 
act against her own preference. Imagine, for instance, 
that you strictly prefer a to b, b to c, but are indifferent 
between a and c. With these preferences, you would be 
rationally obliged to pay a small amount for swapping c 
for b and then pay again for swapping b for a. However, 
once you have acquired a, you have no reason to con- 
tinue swapping. Since you are indifferent between a and 
c, you may keep a, which stops the pump. By keeping 
a, you do not choose an alternative to which another is 
preferred. However, because you are indifferent between a 
and c, your preference violates the transitivity condition. 

The worry outlined above can be further analyzed 
by distinguishing between strong (“forcing”) and weak 
(“nonforcing”) pragmatic arguments (see, e.g., Peter- 
son, 2015). In a strong pragmatic argument, the agent’s 
preferences guarantee that a rational agent will choose 
an alternative to which another is preferred. In a weak 
pragmatic argument, the agent’s preferences permit 
the rational agent to choose an alternative to which 
another is preferred. If you, for instance, strictly prefer 
a to b, and b to c, but are indifferent between a and c, 
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then you are rationally permitted to swap over and over 
again, each time paying a small fee when you swap to 
a strictly preferred option. However, you would not 
be rationally obliged to keep swapping once you reach 
a. Some scholars think that weak pumps are strong 
enough: by merely showing that your preferences permit 
you to be money-pumped, we can conclude that your 
preferences are irrational. Defenders of strong pumps 
insist that for a pragmatic argument to be successful, we 
have to establish that the agent is rationally obliged to be 
money-pumped. 

The distinction between weak and strong pumps is 
also relevant for assessing pragmatic arguments for the 
assumption that rational preferences are complete. It 
is reasonable to think that if you have no preference 
between a and c, then you are rationally permitted, but 
not rationally obliged, to swap. Hence, if you strictly pre- 
fer a to b, and b to c, you would be vulnerable to a weak, 
but not a strong, money pump. 

Some authors have argued that we can justify the 
independence axiom (see section 4.1) by appealing to 
pragmatic considerations. Figure 8.2.1 summarizes a 
sequential choice problem with two choice nodes (boxes) 
and three chance nodes (circles). At the first choice node 
(the leftmost box), the agent chooses between going “up” 
and “down.” If the agent goes up and event -=E occurs (the 
probability of which is 89/100), she wins $0. However, if 
event E occurs (the probability for this is 11/100), the agent 
faces a new choice, in which she can choose to either walk 
away with $1 million or accept a lottery in which she wins 


$1M 


Pr(F) = 10/11 sq 


Pr(-F)= 1/11 $0 


Pr(E) = 11/100 


$0 


$lM+e 
Pr(£) = 11/100 


Pr(-E) = 89/100 
$O+¢ 


Figure 8.2.1 
A decision tree (after Rabinowicz, 1995). 
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$5 million with probability 10/11 and $0 with probability 
1/11. If the agent goes down at the first choice node, she 
faces a lottery in which the outcomes and probabilities are 
exactly as if she were to decide at the outset to go up at 
both the first and second choice nodes, except that a small 
bonus ¢ has been added to each possible outcome. 

The outcomes and probabilities in figure 8.2.1 are 
analogous to those in table 8.2.1. Note that 


x: $1 million; 

y: $5 million with probability 10/11, otherwise $0; 
xpz: $1 million with probability 11/100, otherwise $0; 
ypz: $5 million with probability 10/100, otherwise $0. 


To construct a pragmatic argument for the independence 
axiom, suppose the agent’s preferences are as follows: 


x>y, (1) 


VpZ > xpz. (2) 


For the reason explained in section 4.1, these prefer- 
ences violate the independence axiom. To construct the 
argument, we also assume that the small bonus ¢ in fig- 
ure 8.2.1 does not reverse the agent’s preference between 
ypz and xpz. Given that the bonus is small enough, this 
is not a controversial assumption: 


ypz > Xpz + €> xpz. (3) 


Because the agent prefers ypz to xpz, per (2), she will go 
up at the first choice node in figure 8.2.1 with the inten- 
tion to go down at the second if event E occurs. How- 
ever, if the agent reaches the second choice node, she 
will go up, since she prefers x to y, per (1). According 
to the agent’s preferences, $1 million for sure is worth 
more to her than the risky gamble in which she wins 
$5 million with probability 10/11. However, this means 
that she is willing to forsake a small but certain bonus. If 
the agent goes down at the first choice node, she would 
obtain the same outcome plus the bonus ¢. From a prag- 
matic point of view, agents who act in accordance with 
(1)-(3) thus forsake sure gains, compared to what they 
could have achieved had they instead revised their pref- 
erences in accordance with the independence axiom. 

Is this a convincing pragmatic argument for the inde- 
pendence axiom? To start with, note that the argument 
only purports to show that one of many possible violations 
of the independence axiom causes the agent to forsake 
the small bonus ¢. The argument does not purport to 
show that every possible violation has the same pragmatic 
effect. Suppose, for instance, that the agent prefers x to y 
but is indifferent, or has no preference, between ypz and 
xpz. If this is the case, she is at most vulnerable to a weak 
pragmatic argument but not to a strong one. 
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A more fundamental concern is that the argument 
presupposes a theory of sequential decision-making few 
people find plausible. In essence, the problem is that 
the agent who goes up at the first choice node with the 
intention of going down at the second is overly myopic 
(shortsighted). Schick’s (1986, p. 117) observation that a 
rational agent will see “what is in store for him” seems 
equally applicable here as in the money-pump argument 
for transitivity mentioned earlier. According to Schick, 
a sophisticated agent facing a sequential choice problem 
should reason backward: if the agent were to reach the 
second choice node, she would prefer x to y, and she 
knows this already at the first choice node. Therefore, 
there are only two options open to her: she can (i) go 
down at the first choice node or (ii) go up at the first 
choice node with the intention to go up at the second. 
The third alternative, up followed by down, is not a gen- 
uine alternative. It should thus be deleted from the list 
of options. Therefore, the example in figure 8.2.1 fails to 
show that agents who violate the independence axiom 
will act contrary to their own preference. 

Rabinowicz (1995) argues that it is possible to over- 
come this objection: the sophisticated approach does 
not make the agent immune to pragmatic arguments. 
In figure 8.2.2, the sophisticated agent with the prefer- 
ences described in (1) and (2) would reason backward 
just as before: she prefers $1 million for sure (i.e., x) 
to $5 million with a probability of 10/11 (y), so she 
knows already at the first choice node that she would 
go up at the second choice node. At the first node, she 


$1M 


$5M 


$0 


Pr(E) = 11/100 


Pr(-E) = 89/100 
$0 


Pr(F) = 10/11 $sM—e 


Pr(E) = 11/100 
$0 -e€ 


Pr(-F) = 1/11 


Pr(-E) = 89/100 
$0-e€ 


Figure 8.2.2 
A decision tree for sophisticated agents (after Rabinowicz, 1995). 
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thus faces a choice between (i) going down and (ii) 
going up with the intention of going up at the second 
choice node. Because the agent prefers ypz to xpz, and 
the subtraction of ¢ from the outcomes of ypz does not 
alter her preference, she will go down at the first node. 
However, according to Rabinowicz, down is dominated 
by the plan to go up at the first node and down at the 
second. From a pragmatic point of view, the sophisti- 
cated agent is thus acting contrary to her own prefer- 
ence. She prefers ypz to ypz-e, but the sophisticated 
approach forces her to choose ypz- e. Advocates of the 
sophisticated approach respond to this that the domi- 
nating plan (up, down) is not feasible and can therefore 
be ignored. 

McClennen (1990) rejects the myopic and sophis- 
ticated approaches and proposes what he calls a reso- 
lute approach. Resolute agents adopt a plan at the first 
choice node for how to act at all later choice nodes and 
then stick to the initial plan as long as no new informa- 
tion is received. The plan will typically be conditional: 
“Go up at the second node if event E occurs” and so on. 
Resolute agents who violate the independence axiom 
cannot be money-pumped, but a common objection is 
that resolute choice presupposes perfect self-control (for 
a discussion, see, e.g., Peterson & Vallentyne, 2018). 


6. Causal versus Evidential Decision Theory 


Decision theorists disagree on how rational agents should 
respond to information about causal processes. The locus 
classicus for this debate is a thought experiment first dis- 
cussed in print by Nozick (1969), who attributes it to the 
physicist William Newcomb. 

Imagine a supercomputer that is very good at pre- 
dicting people’s choices by scanning their brains. You 
are asked to choose between two boxes. The first box 
is transparent, and you can see that it contains exactly 
$1,000. The other box is not transparent. It contains 
either $0 or $1 million, depending on how the super- 
computer predicts you will choose. The operator, whom 
you have no reason to distrust, tells you that (i) if the 
supercomputer predicts that you will take only the first 
box, then the operator will put $1 million in the second 
box, but (ii) if the supercomputer predicts that you will 
take both boxes, then the operator will put $0 in the 
second box. The supercomputer’s prediction is highly 
accurate but not infallible. Of the 1,000 predictions the 
computer has made so far, the computer made correct 
predictions 990 times. 

You are now asked to choose between the following 
options: 
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(a) Take box 1 ($1,000) and box 2 (either $0 or $1 million). 
(b) Take only box 2 (either $0 or $1 million). 


Note that the supercomputer first predicts how you will 
choose; then the operator adjusts the content in box 2 by 
putting either $1 million or $0 in the box; and finally, after 
these events have already happened, you make your choice. 
The temporal order is important because it guarantees that 
your choice does not causally influence the content of box 2. 
Causal decision theory (CDT) holds that the only ratio- 
nal option is to take both boxes. Because the operator has 
already put $1 million or $0 in the second box when it 
is time for you to choose, the decision you make will not 
affect the content of the second box. If you take both boxes, 
you will thus get $1,000 in addition to whatever is in the 
second box. Another way to put this is to say that taking 
two boxes dominates taking one box: no matter what the 
world is like ($0 or $1 million in the second box), you will 
be at least as well or better off if you take both boxes (for a 
detailed defense of CDT, see, e.g., Joyce, 1999). 
Advocates of evidential decision theory (EDT) take 
issue with what they consider to be an overly narrow 
analysis offered by causal decision theorists. Their point 
is that the analysis offered by causal decision theorist 
does not accurately reflect the relevant information 
available to the agent. In Newcomb’s problem, you know 
that the probability that the supercomputer’s prediction 
is correct is 99%. This holds true even if you take only 
one box. The choice you make thus counts as evidence for 
what the supercomputer has predicted you will do. The 
only rational option according to advocates of EDT is to 
calculate the expected utility such that probabilities are 
conditional on which act is performed (see table 8.2.3). 
To simplify the calculation, we assume that the agent’s 
utility for money is linear. (Note that we could easily 
modify the prizes to fit any utility function the agent 
may have.) It can then be easily verified that the expected 
utility of taking only the second box exceeds that of tak- 
ing both boxes, meaning that this is the only rationally 
permissible option. 
Which theory do we have most reason to accept? 
Egan (2007) argues that the following example shows 
that CDT sometimes has unacceptable implications: 


Table 8.2.3 
Newcomb’s problem 
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Paul is debating whether to press the “kill all psychopaths” 
button. It would, he thinks, be much better to live in a world 
with no psychopaths. Unfortunately, Paul is quite confident 
that only a psychopath would press such a button. Paul very 
strongly prefers living in a world with psychopaths to dying. 
Should Paul press the button? (p. 97) 


Joyce (2012), an influential advocate of CDT, argues 
that “CDT does not recommend shooting as the uniquely 
rational act. ... Agents who think their way through [this 
example] from the perspective of CDT will wind up being 
correctly indifferent between shooting and refraining 
once they have taken all their causally relevant information 
in to account” (p. 125). It is an open question whether this 
is a satisfactory response to Egan-style examples. Many 
people seem convinced that it is not rationally permis- 
sible to shoot or press the button (see also Ahmed, 2014, 
and chapter 7.1 by Pearl, this handbook). 


7. Paradoxes and Puzzles 


This section reviews some paradoxes and puzzles not 
mentioned in the previous sections. 


7.1 Allais’s Paradox 

The point of departure for Allais’s (1953) paradox, which 
is not a paradox in a strict philosophical sense but rather 
an unintuitive implication of the expected utility princi- 
ple, is the four acts f, g, f', and g’ listed in table 8.2.2 in 
section 4.2. To state the paradox, we calculate the arith- 
metic difference in expected utility between acts f and g, 
as well as between f’ and g’. Since the agent’s utility of 
money is unknown, we write 


u(f)-u(g) = 
= u(1M) - [.01-u(OM) + .1-u(5M)+.89-u(1M)| 
= .11-u(1M)-[.01-u(O) + .1-u(SM)], 
u(f’) - ug") = 
= [.11-u(1M) + .89-u(0)] -[.9-u(OM) +.1-u(5M)] 
= .11-u(1M) -[.01-u(0) +.1-u(5M)]. 


As noted in section 4.2, it seems reasonable to prefer fto g 
but g’ to f’, even though this violates Savage’s sure-thing 
principle. However, the difference in expected utility 
between f and g is exactly the same as that between f[' 


$1 million in second box 


$0 in second box 


Take second box $1 million 
only 
Take both boxes $1 million + $1,000 


(prob. .99) $0 (prob. .01) 


(prob. .01) $1,000 (prob. .99) 
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and g’ regardless of what the agent’s utility of money is. 
Therefore, whenever fis preferred to g, the agent will also 
prefer f’ to g', regardless of the agent’s utility of money. 
This challenge to the sure-thing principle is thus a chal- 
lenge to the principle of maximizing expected utility. 
Regardless of how much or little you care about money, 
you are bound to violate the expected utility principle if 
you prefer fto g but g' to f’. The “paradox” consists in the 
fact that there is no way of assigning utility to money that 
renders the expected utility principle compatible with a 
preference for f over g and a preference for g' over f". 

The standard response to Allais’s paradox is to point 
out that the description of the outcomes in table 8.2.2 is 
incomplete. It does not take into account that you will 
be disappointed if you choose act g and do not become 
a millionaire. A more accurate representation would be 
the one in table 8.2.4. 

Although we do not know exactly what the agent’s 
utility of “$0 and disappointment” is, it seems reason- 
able to assume that it is less than the utility of $0. There- 
fore, the difference in expected utility between fand g is 
not the same as that between f’ and g’. 


7.2 The St. Petersburg Paradox 

The St. Petersburg paradox is one of the oldest and most 
well-known problems in decision theory. It is not a para- 
dox in a strict philosophical sense (just like the other 
problems discussed here) but merely a counterintuitive 
implication of the expected utility principle. 

Imagine that a fair coin is tossed n times until it lands 
heads up for the first time, after which you receive a 
prize worth 2” units on your personal utility scale. If, 
for instance, the coin lands heads on the first toss, you 
win 2 units, and if you get to toss it five times, you win 
2° = 32 units, and so on. How much would a rational 
agent be willing to pay for an opportunity to playing 
this game? 

The expected utility of the St. Petersburg game is 


Bs 1 4,l. 


2+ 8+---=14+14+14+---= 0, 
2 4 8 
Table 8.2.4 
Allais’s paradox 
Ticket 1 Ticket 2-11 Ticket 12-100 
Actf $1 million $1 million $1 million 
Actg  $0and $5 million $1 million 
disappointment 
Actf’ $1 million $1 million $0 
Actg’ $0 $5 million $0 


Hence, a rational agent would, according to the expected 
utility principle, be willing to sacrifice all her assets for 
the opportunity to play this game once. But this seems 
absurd. The most likely outcome of the game is that the 
agent wins 2, 4, or 8 units of utility. Can a remote pos- 
sibility of winning a huge amount really compensate for 
a very likely loss of all one’s assets? 

In Bernoulli's (1738/1954) original formulation of the 
St. Petersburg paradox, the prizes were gold coins rather 
than units of utility. The concept of utility was invented 
by Cramer (1728/1975) as a way of explaining why a ratio- 
nal agent is not required to sacrifice all her assets for the 
opportunity to play the game. If the agent’s marginal value 
(utility) of gold is decreasing, the expected utility of the 
gamble is finite. However, although the notion of decreas- 
ing marginal utility is still important in many economic 
analyses, modern philosophers are aware that the paradox 
can be reformulated in the manner mentioned above, in 
which the agent wins 2” units of utility rather than some 
units of gold. This reformulation of the paradox of course 
presupposes that utility is unbounded (Peterson, 2020). 


7.3 The Pasadena Puzzle 

The gist of the Pasadena Puzzle is that some gambles 
seem to have no expected utility at all. Before we state 
the paradox, it is helpful to recall that some infinite 
series have no unique sum. Consider, for instance, the 


alternating harmonic series (-D"" | We can write this as 
n 
- — Be 2% z +--+. The sum of this series, added up 
2 3 4°55 


in this order, is In 2. However, the Riemann rearrange- 
ment theorem entails that the sum of the alternating 
harmonic series (and other conditionally convergent 
series) will depend on the order in which the terms are 
summed up. In fact, the terms can be rearranged in a 
permutation such that the alternating harmonic series 
converges to any given value, including + and -~. 

To turn this somewhat esoteric mathematical fact 
into an objection to the expected utility principle, 
Nover and Hajek (2004) ask us to imagine that you toss 
a fair coin n times until it lands heads up for the first 
time. If n is odd, you win 2"/n units of value, but if 1 is 
even, you have to pay 2"/n. Do we have any reason to 
think that the expected value of the Pasadena gamble is 


+.--+=I]n2? Recall that we 


could also sum up the terms in some other order that 
would make the series converge to any real number we 
want. Suppose, for instance, that we write down each 
term of the sum on separate pieces of paper and drop all 
of them on the floor before we sum up the terms. Why 
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not just sum up the terms in the order we happen to pick 
them up from the floor? 

Easwaran (2008) responds to this challenge by intro- 
ducing a distinction between weak and strong expected 
value. He shows that the Pasadena gamble’s (only) weak 
expected value is In 2. However, Lauwers and Vallen- 
tyne (2016) remind us that not all gambles have any 
weak expected value. Their suggestion is that we should 
rather think of the entire interval of all possible val- 
ues the series can take as its value. However, a problem 
with this proposal is that it is not clear how we ought to 
choose between something worth, say, [-%, +] and 0. 
The interval [-00, +00] is clearly not worth as much as 0, 
because if it were, we could obviously reduce the value 
of the series to a point value, which would make the 
interval value redundant. 


8. Conclusion 


There is persistent disagreement about how the princi- 
ple of maximizing expected utility should be interpreted 
and justified. None of the positions reviewed in this 
chapter appears to be fully satisfactory. However, despite 
all these problems, it seems likely that the expected util- 
ity principle will continue to dominate debates in deci- 
sion theory for the years to come. There are few, if any, 
promising alternatives to the expected utility principle. 
Buchak’s (2013) recent modification of the principle is 
worth considering, but the decision rule she proposes 
makes the agent vulnerable to pragmatic objections sim- 
ilar to those mentioned in section 5, as she points out 
herself. Kahneman and Tversky’s (1979) prospect theory 
offers reasonable descriptive predictions, but no one 
thinks their theory would fare better than the expected 
utility principle from a normative perspective (see also 
chapter 8.4 by Hill, this handbook). 


Notes 


1. Pascal and Bernoulli discuss the principle of maximizing 
expected monetary value. Cramer’s contribution was to clearly 
state the distinction between monetary value and utility. (He 
used the term “moral value.”) 


2. The law of large numbers comes in two versions, the “weak” 
and the “strong” law of large numbers. The version presented 
here is the weak version. 


3. Note that the first edition of von Neumann and Morgenstern’s 
book, published in 1944, does not include their axiomatization 
of the expected utility principle. The axioms first appeared in an 
appendix to the second edition published in 1947. 


Martin Peterson 
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8.3 Prospect Theory 


Andreas Glockner 


Summary 


Prospect theory is one of the most influential descriptive 
theories of risky choice. In this chapter, the core features 
of the theory are summarized and the theory is contrasted 
against theoretical standards of rationality as defined by 
expected utility theory. Developments of the theory, criti- 
cal debates, and recent findings are presented. Empirical 
evidence overall supports prospect theory as a descriptive 
model of risky choice. The theory can qualitatively account 
for multiple deviations from theoretical rationality, and it 
predicts simple risky choices from given information bet- 
ter than heuristics and other competing models. Findings 
particularly concerning more complex risky choices and 
decisions from experiences, however, also show limita- 
tions of the theory. Prospect theory is mainly considered 
an as-if model that predicts choices only. Findings con- 
cerning the cognitive processes underlying risky choice 
are mixed and warrant further investigation. 


1. Prospect Theory: A Descriptive Theory 
of Risky Choice 


In everyday decisions, individuals often have to select 
between (more or less) risky options for which outcomes 
realize with particular probabilities. Here, “risk” refers to 
the fact that one of several possible states of the world 
can realize, but it is uncertain which. These states of the 
world lead to outcomes that differ concerning value 
or utility for the decision maker. Risky decision can in 
principle concern any kind of objects, and prospects can 
range from highly complex and important decisions, 
such as job selection or company investments, to con- 
sumer decisions for any kind of products (e.g., insur- 
ances, lottery tickets, health states, food). 

According to typical standards of theoretical rational- 
ity as, for example, the economic approaches (Becker, 
1976; von Neumann & Morgenstern, 1947), such deci- 
sions should only be influenced by the subjective utili- 
ties of outcomes and the probabilities of states of the 


world to realize (for details, see chapter 8.2 by Peterson, 
this handbook). Typical empirical investigations of risky 
decisions reduce complexity by focusing on exactly these 
factors. Research thus often abstracts from all further con- 
text factors and focuses on the selection between gambles 
with outcome and probability information provided. An 
example would be the choice between gambles A and B 
for which A pays €4 with certainty and B pays €10 with 
probability .50 and €0 otherwise. Prospect theory (Kahne- 
man & Tversky, 1979; Tversky & Kahneman, 1992) is the 
most prominent descriptive theory for decisions between 
monetary gambles and also for risky choice outside the 
lab (Camerer, 2005; Kahneman, 2003; Rabin, 2003). 

Of particular interest are regularities captured by pros- 
pect theory that deviate from the predictions of a strict 
interpretation of theoretical rationality as introduced 
above. In this chapter, I review prospect theory and recent 
findings and discuss relations between prospect theory 
and rationality. 


2. Core Features of Prospect Theory 
and Important Findings 


Prospect theory contains four core features: (i) reference 
dependence, (ii) diminishing sensitivity of outcomes 
relative to the reference point, (iii) loss aversion, and (iv) 
a nonlinear probability weighting function with over- 
weighting of small probabilities and underweighting of 
large probabilities (figure 8.3.1; more detailed explana- 
tions for all four features below). 

Specifically, the theory predicts (i) “that the carriers 
of value are changes in wealth or welfare, rather than 
final states” (Kahneman & Tversky, 1979, p. 277). This 
means that individuals perceive outcomes (or final 
wealth states) not in absolute terms but as changes 
relative to a reference point. The subjective utility of 
an option (which Kahneman and Tversky refer to as 
“value,” V) is a function of the gains and losses relative 
to the reference points and their probabilities. The exact 
way in which outcomes are transformed into subjective 
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Value function 


U(o,r) 


sf 
O=r 0-r 
——-—- losses gains 


Figure 8.3.1 


Andreas Gléckner 


Probability weighting function 


Prospect theory functions (examples) for transforming outcomes o into utilities U(o,r), taking into account a reference point r (left), 


and probabilities p into decision weights w(p) (right). 


utilities is described by a value function with a kink at 
the reference point (figure 8.3.1, left). It is assumed that 
outcomes are considered and transformed in isolation, 
instead of being integrated with other outcomes or prob- 
abilities first. 

Prospect theory assumes (ii) a concave value function 
in the gains and a convex value function in the losses, 
reflecting the principle of diminishing sensitivity of dif- 
ferences from the reference point (figure 8.3.1, left). From 
this functional form, it follows that individuals should be 
risk averse in the gain domain and risk seeking in the loss 
domain.’ 

The theory predicts (iii) that people are loss averse, 
which means that they are more averse to Josses relative to 
their reference level than they are attracted to same-sized 
gains. This is reflected by the steeper value function in the 
losses as compared to the gains in figure 8.3.1 (left). 

Finally, prospect theory predicts that (iv) outcomes 
are not weighted by their objective probabilities but by 
decision weights, which (according to more recent ver- 
sions of the theory) follow an inverse S-shaped function 
(i.e., decision weights regress to the mean for extreme 
probabilities: figure 8.3.1, right; Tversky & Kahneman, 
1992). Outcomes with low probabilities receive increased 
weight (i.e., decision weights are higher than the respec- 
tive probabilities), whereas outcomes with high proba- 
bilities receive reduced weight (i.e., decision weights are 
lower than the respective probabilities). 

Prospect theory can account for a multitude of find- 
ings concerning risky choices in the field (for a review, 


see Camerer, 2005) and also in the lab. Overweighting of 
small probabilities, for example, can explain why people 
buy insurances and lottery tickets although their price is 
higher than their expected value and/or expected util- 
ity. Loss aversion can explain why stock returns are too 
high relative to bond returns (equity premium) and why 
people demand higher prices for selling a product they 
own as compared to their willingness to pay for the same 
product (endowment effects). Diminishing sensitivity 
can explain why people sell winning stocks too early (risk 
aversion in the gain domain) and hold losing stocks for 
too long (risk seeking in the loss domain) and why they 
shift to long shots in horse betting at the end of the day. 
One of the most influential demonstrations of devia- 
tions from rationality predicted by prospect theory are 
framing effects (Tversky & Kahneman, 1981). In typical 
framing studies, choice problems are presented in dif- 
ferent contexts (frames), while holding the normatively 
relevant factors (probabilities and outcomes) of the 
options constant. In the classic Asian disease study, for 
example, substantial shifts in preferences resulted from 
presenting two programs to cure an Asian disease either 
in a loss frame (by mentioning the persons who would 
be killed) or in a gain frame (by mentioning the remain- 
ing persons who would survive). In the loss frame, most 
people were risk seeking and preferred the risky pro- 
gram, a program in which all people could be killed 
with some probability. In the gain frame, in contrast, 
individuals were risk averse and preferred the safe pro- 
gram that assured the survival of some people (Tversky 
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& Kahneman, 1981). The very robust finding (e.g., Klein 
et al., 2014) that a mere reframing of what were objec- 
tively the same options influences choice, constitutes a 
violation of invariance that “cannot be defended as nor- 
mative” (Kahneman, 2003, p. 163). 


3. Normative Theories of Risky Choice 
and Prospect Theory 


Descriptive theories of decision making, such as prospect 
theory, aim to describe and predict how people make 
decisions. Normative theories, in contrast, describe how 
decisions should be made and define a standard of ratio- 
nality against which such descriptive models are typi- 
cally evaluated. Normative theories have to be defined 
with respect to the goals that a decision maker aims to 
achieve. Not taking all goals of a person into account 
can thus lead to wrong standards (cf. chapter 9.4 by 
Dhami & al-Nowaihi, this handbook). 


3.1 Standard Rational Models: Expected Value 

and Expected Utility Theory 

If the decision maker’s only goal is to maximize the 
average (or sum) of payoffs over an infinite number of 
decisions, then she should choose the option that max- 
imizes expected value (EV). Here, EV is defined as the 
sum of lottery outcomes (payoffs) 0; multiplied by their 
probabilities p; for all possible outcomes i: EV = ;p;0;. In 
the above example, according to EV, the risky gamble 
B(EV =0.5x€10 = €5) should be chosen over the safe 
gamble A (EV = €4). 

This picture has to be qualified for at least three rea- 
sons: utility might not be linearly related to monetary 
outcomes, individuals may have further or different 
goals than maximizing the average of payoffs, and deci- 
sions are not infinitely repeated. All three issues have 
been (or at least can be) taken into account in expected 
utility (EU) theory to develop a normative model for 
risky choice that goes beyond EV. 

According to one common implementation of EU the- 
ory for monetary gambles, individuals choose the option 
that maximizes EU =}, p,o%, with p; being the probabil- 
ity of outcome 9; to realize and 0<a<1. In the gamble 
example introduced above, for a person with a reasonable 
sensitivity to outcomes (often also referred to as “tisk aver- 
sion”) of a = 0.7, choosing the safe gamble A (EU = 2.64) 
over the risky gamble B (EU = 2.51) in a one-time decision 
is rational according to this implementation of EU theory. 
The form of this EU value function is concave, which was 
originally suggested by Bernoulli (1738/1954) to account 
for the St. Petersburg paradox.’ The EU value function 


considered here is equivalent to the value function of 
prospect theory when considering the gain domain only 
(figure 8.3.1, left, upper-right sector). Hence, EU theory 
assumes a diminishing marginal utility of money. That is, 
raising the wealth level of an individual by a particular 
amount (e.g., €1,000) leads to a larger change in utility 
than adding the same amount once again (e.g., adding 
another €1,000). 

Importantly, in contrast to prospect theory, EU the- 
ory does not assume that utility is reference-dependent. 
Outcomes are assessed relative to a wealth level of zero. 
As a consequence, according to EU theory, there is no 
qualitative differentiation between gains and losses, and 
they should be treated the same (i.e., no reflection of 
the utility curve at the reference point and therefore no 
risk seeking in the losses; no loss aversion). Finally, EU 
theory assumes that probabilities are used without any 
transformation in weights. The core difference between 
EV and EU theory is that the latter assumes a value func- 
tion that transforms outcomes into utilities (in this case 
by raising them to the power of a). 

Individuals often not only want to maximize the sum 
of outcomes over a long or infinite number of gambles 
but also have further goals. In some situations, for 
example, it can be rational to be short-sighted and to 
think about the current situation in isolation, not tak- 
ing into account potentially higher future outcomes. It 
will be reasonable for individuals to avoid falling below 
a minimum wealth level in order to keep a decent life, 
and short-sightedness might be extremely rational for 
an individual to ensure survival. More complex further 
goals are possible too. Individuals want to maximize 
joint outcomes, maximize the difference in wealth com- 
pared to others (Van Lange, 1999), or avoid sticking out 
from the crowd. These further goals can, in principle, be 
accounted for in EU theory by incorporating them (e.g., 
as further preferences) in individuals’ utility functions 
(see chapter 9.4 by Dhami & al-Nowaihi, this handbook). 


3.2 Prospect Theory and Rationality 

Tversky and Kahneman (1981) used EU theory as a nor- 
mative (rational) standard for risky choice by stating, 
“When faced with a choice, a rational decision-maker 
will prefer the prospect that offers the highest expected 
utility” (p. 453). They then demonstrated that behavior 
systematically deviates from these standards. Prospect 
theory, a descriptive decision theory, aims to account for 
individuals’ actual choice behavior, which often follows, 
but in some cases systematically deviates from, these 
rational standards. The theory does not aim at provid- 
ing new standards of rationality; quite the contrary, 
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the description and prediction of deviations from these 
standards lie at the heart of the theory. 


4. Theory Development and Versions 
of Prospect Theory 


The original version of prospect theory (Kahneman & 
Tversky, 1979) contained two phases. In the “editing 
phase,” a preliminary analysis of the offered prospects is 
conducted. The editing often yields a simpler representa- 
tion of the prospects, for example, by discarding shared 
outcomes, combining probabilities of equal outcomes, or 
coding outcomes as gains and losses. In the subsequent 
“evaluation phase,” the edited prospects are evaluated, 
and the option with the highest utility is chosen. The 
original version did not contain a formal description of 
the probability weighting function. Cumulative prospect 
theory, which was introduced in 1992, included fully 
specified functions for outcomes and probability weight- 
ing, as depicted in figure 8.3.1 (Iversky & Kahneman, 
1992). Furthermore, the specification of the probability 
weighting function included a specification of outcome- 
rank-dependent probability weights and a cumulation 
principle to ensure that probability weights add up to 1. 
Therefore, probability weights are influenced by the size 
of the outcome they are attached to relative to the other 
outcomes.* Hence, cumulative prospect theory predicts 
not only nonlinear probability weighting but also that 
the same outcome with the same probability receives dif- 
ferent weights, depending on the size of the other out- 
comes, which is in conflict with normative models such 
as EU theory. In the version suggested in 1992, cumula- 
tive prospect theory contains five parameters: 


* a: the sensitivity to outcomes in the gains (it deter- 
mines the curvature of the value function in the gains; 
figure 8.3.1, left, upper-right sector), 

* 6: sensitivity to outcomes in the losses (it determines 
the curvature of the value function in the losses, 
which is additionally scaled by A; figure 8.3.1, left, 
lower-left sector), 

* A: loss aversion (it captures the relative increase in 
sensitivity to losses, that is, the factor by which 
losses loom larger than equal gains, when assuming 
a= p), 

* y: probability weighting in gains (it determines the 
shape of the probability weighting function for gains, 
that is, the sensitivity to probabilities in gains), 

* 6: probability weighting in losses (it determines the 
shape of the probability weighting function for losses, 
that is, the sensitivity to probabilities in losses). 
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Since the exact formalization of cumulative prospect 
theory is not of particular interest for the current chap- 
ter, it is not provided here, and I refer the reader to Tver- 
sky and Kahneman (1992) for a description of the full 
formalization of the theory. 

Further extensions concerned, for example, modifica- 
tions to the probability weighting function (for an over- 
view, see Stott, 2006); the introduction of a two-stage 
model of prospect theory, taking into account in a first 
step probability estimation in decisions under uncer- 
tainty (i.e., without explicit probability information avail- 
able) based on support theory (Fox & Tversky, 1998); and 
an extension that allows reference points to be uncertain 
too (i.e., lotteries as reference points; Schmidt, Starmer, & 
Sugden, 2008). Overall, prospect theory inspired several 
extensions and theoretical developments, but the core fea- 
tures generally remained (for an excellent overview, see 
Wakker, 2010). 


5. Empirical Findings 


The capability of prospect theory to account for risky 
choice behavior has been critically investigated using 
various methodological approaches. A full review would 
be beyond the scope of this chapter. I will exemplify some 
of the methodologies, debates, and crucial findings. 


5.1 Predictive Ability 

In the original paper, Kahneman and Tversky (1979) pre- 
sented results for 12 selected decision problems in which 
the majority of participants chose the option that was 
predicted by prospect theory and contrary to EU theory. 
However, these problems were explicitly constructed 
to disprove EU theory. It remained unclear whether 
prospect theory is indeed a good theory for predicting 
choices also in a less selective set of decision problems. 
Furthermore, it has been argued that cumulative pros- 
pect theory (due to its free parameters) might only be 
able to fit data post hoc but that simpler models, such 
as the priority heuristic,” might be better at predicting 
choice, due to the overfitting of prospect theory (Brand- 
statter, Gigerenzer, & Hertwig, 2006). 

We put this question to a direct test in an adversarial 
collaboration with a proponent of the heuristic perspective 
(Gléckner & Pachur, 2012). Participants made incentiv- 
ized decisions in a large and diverse set of gamble prob- 
lems at two points in time. Importantly, the set of gamble 
problems was intensely discussed a priori to avoid giving 
any model a strategic advantage. The competing models 
included various versions of prospect theory, EU theory, 
EV theory, and all existing simple heuristics that we 
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were aware of. All models were fit to the data at time 1 
to predict choices at time 2. The results showed that nei- 
ther priority heuristic nor any of the other heuristics—or 
their adaptive use as specified in some models—could 
predict choices nearly as well as even a simple EV the- 
ory. EV theory in turn predicted significantly worse than 
EU theory. The best predictions by far were achieved by 
cumulative prospect theory. Among different versions, 
the model without any free parameters using Kahneman 
and Tversky’s parameter values from 1992 did fairly well, 
but individual-level fitting improved predictive ability 
further and close to the maximum level possible given 
limited reliability in choices. 

Overall, this and many further investigations, includ- 
ing analyses in applied settings (Camerer, 2005), show 
that prospect theory is a model well suited for describing 
risky choice behavior. Given this excellent performance 
as a descriptive theory, prospect theory seems particularly 
relevant as a starting point for reflecting about extended 
standards of practical rationality. Prominent alternative 
models seem less relevant for deriving such extended 
standards, considering their typically worse performance 
as descriptive models. For example, given that no single 
heuristic (simplified decision strategy) and no model of 
adaptive strategy usage exists that can predict behavior 
in the most basic gambling paradigm nearly as well as 
prospect theory (Gléckner & Pachur, 2012), such models 
might be less useful for deriving extended standards of 
rationality (but see Gigerenzer & Brighton, 2009, for a 
different position). 

As a limitation, however, it remains unclear whether 
this superiority of cumulative prospect theory in pre- 
dicting simple choices between two lotteries with two 
outcomes each and explicitly provided probabilities gen- 
eralizes to more complex risky choice tasks, for which 
heuristics might be more prevalent (see below). 


5.2 Critical Property Testing 

A second approach to test cumulative prospect theory 
against alternative models is to investigate critical prop- 
erties of the theory that are predicted under any set of 
parameters. Birnbaum conducted intense empirical inves- 
tigations following this approach, particularly taking into 
account more complex lotteries with three outcomes. He 
demonstrated multiple critical choice anomalies that pros- 
pect theory could not account for but alternatively sug- 
gested models such as the transfer of attention exchange 
model (Birnbaum, 2006, 2008). Birnbaum showed that 
splitting the most (least) attractive branches of a gamble 
makes them more (less) attractive and can lead to prefer- 
ence reversals that prospect theory cannot account for. 
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For example, most individuals prefer gamble A: .85 to win 
$100 and .15 to win $50, over gamble B: .95 to win $100 
and .05 to win $7 (Birnbaum, 2008, Table 1). This (major- 
ity) preference, however, reverses if the less attractive 
outcome is split for gamble A (.85, $100; .10, $50; .05, 
$50) and the more attractive outcome is split for gamble 
B (.85, $100; .10, $100; .05, $7), although the games 
remain substantially equivalent. 

Similarly, it was shown that in gambles with multi- 
ple winning and losing outcomes, choices are more in 
line with a p-win heuristic (i-e., choose the option that 
maximizes the overall probability of winning outcomes) 
and could not be accounted for well by prospect theory 
(Payne, 2005). These investigations show limitations 
of prospect theory concerning its ability to account for 
more complex decision tasks (i.e., ones with more than 
two outcomes). Given the higher task complexity, it is 
psychologically plausible that behavior is less in line 
with prospect theory in more complex tasks due to limi- 
tations of deliberate cognitive capacity. Further investi- 
gations are needed to test whether the average predictive 
ability of prospect theory in a broad set of more complex 
tasks falls below the performance of competing theories. 


5.3 Decisions from Experience 

The standard paradigm for investigating prospect theory 
relies on decisions from descriptions, that is, decisions 
between gambles with explicitly provided probabilities 
(see examples above). Many everyday decisions, however, 
involve decisions from experience, in which probabili- 
ties are not explicitly provided but have to be inferred. 
This has led to a large literature on ambiguity (unknown 
probabilities) in the economic literature (Ellsberg, 1961), 
surveyed by Trautmann and van de Kuilen (2015) and 
Wakker (2010). I will focus on another stream of litera- 
ture, initiated by Erev, Hertwig, and colleagues (Barron & 
Erev, 2003; Hertwig, Barron, Weber, & Erev, 2004). These 
authors showed substantial differences between deci- 
sions from experience and decisions from description, 
the so-called description—experience gap. In typical stud- 
ies on decisions from experience (Hertwig et al., 2004), 
individuals are allowed to sample outcomes as long as 
they wish, before making a consequential choice.° For 
example, an individual might sample five outcomes 
from gamble A (€4,€0,€4,€4,€4) and five outcomes 
from gamble B (€3, €3, €3, €3, €3) and then decide to stop 
search and select gamble A. Hertwig et al. (2004) showed, 
for example, that for a choice between gamble A: .8, €4; 
.2, €0, and gamble B: €3, 1.0 (i.e., with certainty), most 
individuals choose gamble B in decisions from descrip- 
tions but gamble A in decisions from experience. 
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One important factor that contributes to this differ- 
ence in choice behaviors are sampling biases. Of particular 
importance here is the bias that results from the fact that 
rare events are more likely to be underrepresented in small 
samples due to the skewness of the binomial distribution. 
That is, small samples have a higher likelihood of contain- 
ing too few rare events than of containing too many.’ 

It has been controversially debated whether a new 
theory has to be developed for decisions from experi- 
ence (Hertwig & Erev, 2009) or whether choices can be 
explained by sampling biases plus prospect theory (Fox 
& Hadar, 2006). While the former perspective assumes 
that cognitive processes might be entirely different in 
decisions from experience, proponents of the latter per- 
spective assume that individuals merely sample outcomes 
to estimate probabilities. These estimates are, however, 
influenced by sampling biases, which in turn lead to dif- 
ferent choices in experience-based decisions. 

A large-scale prediction competition (Erev et al., 
2010) showed that a psychologically not very plausible 
“ensemble” (mixture) of models predicted choices in 
experience-based decision making best. Several further 
process models were developed and tested (Barron & 
Erev, 2003; Gonzalez & Dutt, 2011; Lejarraga, Dutt, & 
Gonzalez, 2012). Amore recent model comparison, taking 
into account the data from the Erev prediction competi- 
tion and additional data (Glockner, Hilbig, Henninger, 
& Fiedler, 2016), provided support for prospect theory as 
a model for decisions from experience too. It was found 
that a two-stage model (cf. Fox & Hadar, 2006; Fox & 
Tversky, 1998), assuming (i) probability estimations that 
are influenced by sampling biases and (ii) an application 
of prospect theory based on these probability estima- 
tions, could explain choice behavior very well and better 
than important alternative process models (Gonzalez & 
Dutt, 2011; Lejarraga et al., 2012). 

Furthermore, it was shown that the initially assumed 
direction of the description-experience gap, a qualita- 
tive difference in that rare events are overweighted in 
description but underweighted in experience, does not 
hold universally (Gléckner et al., 2016; Kellen, Pachur, 
& Hertwig, 2016). The assumed gap is highly problem 
dependent, as confirmed by a large-scale meta-analysis 
(Wulff, Mergenthaler-Canseco, & Hertwig, 2018). Specif- 
ically, after accounting for sampling bias® and when tak- 
ing into account a more representative sample of tasks, 
the description—experience gap reverses (Gléckner et al., 
2016; Kellen et al., 2016). That is, when considering the 
outcomes that people have truly experienced by sampling, 
they overweight small probabilities even more strongly 
in decisions from experience than in decisions from 
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description. This finding is in line with much research 
showing that smaller probabilities tend to be overesti- 
mated due to a universal regression-to-the-mean effect 
(K. Fiedler & Unkelbach, 2014). 

Overall, this research highlights that sampling biases 
and also further (partially task-dependent) factors have 
to be taken into account in modeling decisions from 
experience. Given the complexity of the findings (for an 
excellent summary, see Wulff et al., 2018), it is up to fur- 
ther research whether adaptations to cumulative pros- 
pect theory are sufficient to account for decisions from 
experience or whether qualitatively different models 
have to be developed. In a first approximation, a model 
based on sampling bias and cumulative prospect theory 
does surprisingly well also in predicting decisions from 
experience (in the sampling paradigm). 

One important distinction in experience-based deci- 
sion making concerns the differentiation between the 
sampling paradigm (discussed so far) and a feedback 
paradigm. In the feedback paradigm, each sampled out- 
come from a gamble is paid, whereas in the sampling 
paradigm, only the outcome of the final choice is paid. 

Considerable differences between the two paradigms, 
as well as between them and the standard decision-from- 
description paradigm, indicate that dependent on the 
requirements of the situation, people do not focus on 
the current choice task only. Instead, individuals seem 
to adapt—at least partially—to whether adopting a nar- 
rower or a broader perspective, by taking into account 
the outcomes of further decisions, is appropriate. This 
should be considered in future theoretical developments. 


5.4 Process Measures 
Aside from the analysis of choice behavior, further mea- 
sures of, for example, information search and/or response 
time are used to test hypotheses concerning the underlying 
cognitive processes (Schulte-Mecklenbeck, Kiihberger, & 
Johnson, 2019). Various such process measures have been 
applied to risky choice to determine whether and how 
prospect theory might be implemented computationally 
and whether it is a reasonable model at all. Similarly to the 
classic bounded rationality critique of EU theory (Simon, 
1955), it is unreasonable to assume that individuals delib- 
erately apply complex multiplication operations to imple- 
ment prospect theory. Hence, prospect theory is usually 
considered an as-if model instead of a process model. 
Classic studies on processes in risky choice (Payne, 
Bettman, & Johnson, 1988) used the computer-based 
information search paradigm Mouselab. In Mouselab, 
individuals can reveal pieces of information (outcomes 
and probabilities) concerning multiple complex lotteries 
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by moving the mouse to the respective information 
boxes. Payne et al. (1988) have demonstrated that in 
complex risky choice paradigms (multiple options with 
multiple outcomes each), manipulations of the environ- 
mental structure and time pressure induce changes in 
information search. This indicates that individuals also 
change their decision strategies. Payne et al. show that 
these changes are adaptive to the structure of the environ- 
ment, in that individuals tend to change to strategies that 
are more successful in the respective environments. 

The findings by Payne et al. preclude that (all) individu- 
als search for all pieces of information and use prospect 
theory to evaluate them. Results can be better explained 
by assuming that individuals are adaptive decision mak- 
ers and switch, for example, from more complex to sim- 
pler and less effortful strategies (i.e., simple heuristics that 
ignore large parts of the information) dependent on task 
demands (Payne, Bettman, & Johnson, 1993). These results 
converge with the findings from critical property testing 
(see section 5.2) indicating that prospect theory’s ability 
to account for choices in more complex tasks might be 
limited. 

Eye-tracking studies provide further insights concern- 
ing the potential processes underlying risky choices in 
less complex tasks. In these studies (S. Fiedler & Gl6ck- 
ner, 2012; Franco-Watkins & Johnson, 2011; Gl6ckner 
& Herbold, 2011), it was repeatedly shown that indi- 
viduals (i) look up all pieces of information, (ii) mainly 
search information within gambles (instead of compar- 
ing between gambles), and (iii) show choices that are 
in line with a compensatory integration of outcomes 
and probabilities. Thereby, (iv) response times are too 
short to allow for an integration of outcomes and prob- 
abilities by deliberate calculation. Also, (v) individuals 
look up information with short fixations, while delib- 
erate calculations are accompanied by long fixations. 
These findings are hard to reconcile with the assump- 
tion that—even for simple risky choice tasks—weighted 
sums of (transformed) outcomes and probabilities are 
deliberately calculated. The processes might be better 
accounted for by evidence accumulation models (Buse- 
meyer & Townsend, 1993; Usher & McClelland, 2004) 
or parallel constraint satisfaction models (Gloéckner & 
Herbold, 2011; Holyoak & Simon, 1999), which allow 
approximating weighted integration of probabilities and 
outcomes by psychologically plausible processes. 


6. Summary and Future Directions 


Prospect theory is arguably the most influential descrip- 
tive model for risky choice. Prospect theory assumes (i) 
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reference dependence, (ii) diminishing sensitivity of out- 
comes relative to the reference point, (iii) loss aversion, 
and (iv) a nonlinear probability weighting function with 
overweighting of small probabilities and underweight- 
ing of large probabilities. Prospect theory predicts multi- 
ple deviations from expected utility theory, the standard 
normative model of theoretical rationality. These predic- 
tions have received ample support in the lab, and the 
theory can explain deviations from theoretical rational- 
ity that are observed in the field. 

As a descriptive model of choice, prospect theory does 
not aim to define new standards of rationality. Quite the 
contrary, Kahneman and Tversky developed the theory 
to (among other things) account for deviations from 
these standards, such as violations of the invariance 
principle as demonstrated by framing effects. 

A review of recent findings and debates indicated that 
prospect theory is still one of the best models for pre- 
dicting simple decisions from descriptions. For complex 
tasks, the evidence is more mixed. In risky choice between 
options with multiple outcomes, evidence from critical 
property testing and process tracing converges to show 
that individuals rely on different, potentially more heu- 
ristic processes. For decision from experience, sampling 
biases in probability estimation and subsequent applica- 
tion of prospect theory can account for choice behavior 
surprisingly well. Still, in both areas, more research is 
needed to clarify the issues and to develop and compara- 
tively test improved models. 

Process measures for simple decisions from descrip- 
tions indicate that individuals do not apply prospect 
theory by deliberately calculating weighted sums of (trans- 
formed) outcomes and probabilities—even if choices are 
in line with the theory’s predictions. This highlights the 
status of prospect theory as an as-if model. To integrate 
(transformed) outcomes and probabilities, individuals 
seem to rely on qualitatively different processes, which 
also have to be explored in future research. 


Notes 


1. When taking into account the value function and the prob- 
ability weighting function (described below) simultaneously, a 
more complex fourfold pattern of risk attitudes emerges (i.e., 
risk aversion for gains and risk seeking for losses of high prob- 
ability, as well as risk seeking for gains and risk aversion for 
losses of low probability; Tversky & Kahneman, 1992). 


2. According to cumulative prospect theory (Tversky & Kahne- 
man, 1992), decision weights are furthermore dependent on 
the rank of the outcome, and it is assured that they add up to 1 
(details below and in note 4). 
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3. The paradox is that individuals are not willing to pay a 
particularly high amount of money for playing the follow- 
ing St. Petersburg game, which has an infinitely large EV (for 
a detailed discussion, see also chapter 8.2 by Peterson, this 
handbook): a coin is flipped repeatedly until heads appears. 
If, on the first coin flip, tails appears, the person receives €1 
(otherwise zero). The payment amount doubles with each fur- 
ther coin flip landing on tails. When heads appears, the game 
ceases and the player receives the amount determined at the 
previous coin flip (i.e., only this one amount is paid). See also 
section 7.2 of chapter 8.2 by Peterson (this handbook). 


4. This is realized by outcomes being sorted by magnitude sepa- 
rately for gains and losses. The decision weight for the highest 
outcome in the gains (and the lowest outcome in the losses) 
is directly determined according to the probability weighting 
function w (figure 8.3.1, right), whereas the weights of the 
subsequent outcomes are determined as differences in weights. 
For example, consider a gamble with three outcomes O;: .20, 
€20; Oz: .40, €10; O3:.40, €5. The probability weight of O, is the 
weight w(p;) with p;=.20 being transformed according to func- 
tion w. The weight for O2 is the difference between the weights 
for w(p1 + p2) with pi + p2=.60 and w(p,). The weight for O3 is the 
difference between weights w(p: + p2 + p3) with pi + p2+p3 = 1 and 
W(pi+p2). Note that in this example, the decision weight for 
O> will typically be lower than for O3, although both outcomes 
have the same probability of .40. 


5. According to the priority heuristic, individuals apply a stepwise 
noncompensatory comparison of specific attributes of gambles, 
namely, (i) worst outcomes, (ii) probabilities of worst outcomes, 
and (iii) best outcomes with particular difference thresholds. 


6. This describes the sampling paradigm. In the also-often-used 
feedback paradigm, the outcome of each draw is paid (Barron & 
Erev, 2003). Differences are discussed at the end of this section. 


7. In our example, the likelihood of the rare event from gamble 
A (.2, €0) to be underrepresented (i.e., experiencing €0 in less than 
2 out of 10 samples) is .38, while the likelihood of it being over- 
represented in the sample (i.e., experiencing €0 in more than 2 
out of 10 samples) is .32, according to the binomial distribution. 


8. Realized, in this case, by considering only the outcomes that 
people had indeed experienced and not the hypothetical prob- 
abilities of the gambles. 
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8.4 Decision under Uncertainty 


Brian Hill 


Summary 


A series of famous examples casts doubt on the standard, 
Bayesian account of belief and decision in situations of 
considerable uncertainty. They have spawned a signifi- 
cant literature in economics and, to a lesser extent, phi- 
losophy. This chapter surveys some of this literature, with 
an emphasis on the normative issue of rational decision. 


1. The Challenge of Uncertainty 


Imagine that you are faced with two urns, each contain- 
ing only black and white balls. For one of the urns (the 
unknown urn), that is all you know; for the other (the 
known urn), you have counted the balls and know that 
exactly half are black. What would you say if asked for 
the probability that the next ball drawn from the known 
urn was black? And what about the unknown urn? And, 
if you had to place a bet on the next ball drawn from one 
of the urns being black, would you bet on the known or 
the unknown urn? And what if the bets were on the next 
ball being white? 

This example crops up in several disciplines. Keynes 
(1921/2004) uses it to motivate his notion of weight of 
evidence: “It is evident that in either case the probability of 
drawing a white ball is 4%, but that the weight of the argu- 
ment in favor of this conclusion is greater [for the known 
urn]” (p. 75). Replacing direct observation of the com- 
position of the known urn with sampling yields some- 
thing close to Popper's “paradox of ideal evidence.” In 
economics, the example is associated with Ellsberg (1961) 
and has spawned a significant literature on decision under 
uncertainty’ over the past 30 years. 

Its importance lies in the challenge it poses to the 
standard account of rational belief and decision, Bayes- 
ianism (see chapter 8.2 by Peterson, this handbook), and 
in particular to 


* the Bayesian thesis about rational belief: It can be 
represented by a function assigning a single number 


(between 0 and 1) to each proposition or event, which 
satisfies the laws of probability; 


* and the Bayesian thesis about rational decision: The 
chosen action in any decision is that which maxi- 
mizes the expected utility or desirability on the basis of 
the agent’s beliefs. 


Although not formulated as such, Keynes’s and Pop- 
per’s points appear to challenge the first thesis: repre- 
senting belief by probabilities cannot, allegedly, capture 
the weight of evidence supporting a belief. As often noted, 
this argument is not watertight: there are several differ- 
ences between one’s Bayesian probabilities concerning 
the two urns, including one’s beliefs about their compo- 
sition or the robustness of beliefs to further observations 
(e.g., Joyce, 2005). 

By contrast, Ellsberg’s version involves decision. He 
observed a tendency (borne out in subsequent experi- 
ments; Camerer & Weber, 1992) to prefer betting on the 
ball drawn from the known urn over betting on the one 
drawn from the unknown urn, whatever the color—a 
pattern of behavior that has come to be known as uncer- 
tainty aversion or ambiguity aversion. (The field often quali- 
fies the unknown urn as involving ambiguity or Knightian 
uncertainty.”) Such preferences conflict with the Bayesian 
theses. A Bayesian would choose to bet on black from the 
known urn (call this event B,) over black in the unknown 
urn (B,) only if p(Bi) > p(Bu), where p represents her beliefs. 
However, she would have the same preference over bets 
on white only if p(Wx) > p(W.). Clearly, no probability 
function p can satisfy both inequalities. So the Bayesian 
must condemn these so-called Ellsberg preferences as irra- 
tional. Some find this drastic. This opens up the question 
of what more permissive account should or can replace it. 

Although distinct, the points concerning belief repre- 
sentation and decision are intimately related. The dif- 
ference in your ignorance about the two urns—or in the 
“weight of evidence” in the two cases—seems to justify a 
preference between them (Ellsberg, 1961; Levi, 1986; Gil- 
boa, Postlewaite, & Schmeidler, 2009). Indeed, Ellsberg’s 
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point about decision adds a twist to the debate about 
weight of evidence: whatever ways there are of capturing 
something like weight in the framework of Bayesian prob- 
ability, they remain irrelevant for choice. Indeed, the two 
criticisms are perhaps strongest when combined: Bayes- 
ianism, it appears, reserves no role for the weight of evidence 
(or similar factors) in choice. 

This survey will present some of the main responses to 
this challenge, particularly in the economic field of deci- 
sion under uncertainty, but also in philosophy. While 
the focus will be on decision, the relationship with belief 
representation means that this issue cannot be ignored. 
Indeed, it will structure the survey: different belief repre- 
sentations (replacing Bayesian probabilities) naturally 
require different, although not unrelated, decision rules 
(replacing expected utility). The emphasis will be solely 
on rational decision, although the reader should bear in 
mind that some proposals were developed with other 
goals in mind (e.g., tractability for economic modeling 
or descriptive accuracy). For ease of exposition, we shall 
largely eschew technicalities, at times abstracting liber- 
ally from precise formulations to focus on the gist. For 
details, discussions of dimensions other than the norma- 
tive one, or more on the relevant economic literature, 
the reader is referred to excellent and more complete 
existing surveys such as Gilboa and Marinacci (2013) 
and Machina and Siniscalchi (2014). 


Table 8.4.1 
Terminology and definitions 


Terminology Explanation 

S The set of states of the world (state 
space) 

x The set of consequences (conse- 
quence space) 

f, & ete. Acts (objects of choice): functions 
from S to X 

E, F, etc. Events: subsets of S 

A The set of probability measures over 
the state space S (probability space) 

EU, f The expected utility of act f 
calculated with probability 
measure p € A and utility function 
u: EU,f = X p(s)u(f(s)) 

seS 

1, 0, x, Consequence yielding utility 
value 1,0,x,... 

1,0 Bet yielding consequence 1 if event E 


occurs and consequence 0 otherwise 


The functional V rep- 
resents the preference 
relation >. 


For every pair of acts f, g: f= gif and 
only if V(f) = V(g). 


Brian Hill 


Table 8.4.1 lists the main terminology and defini- 
tions used throughout the chapter, which are standard 
in the economic branch of decision theory (although 
little depends on the use of this framework). For simplic- 
ity, we assume a fixed utility function throughout the 
chapter and use 1, 0, x to refer to consequences yielding 
utility 1, 0, x, respectively, according to this function. 
We assume sets to be finite, as far as possible (so as to use 
sums instead of integrals). Some notions are illustrated 
on the previous example in table 8.4.2. The columns 
correspond to the states of the world, each specifying 
the resolution of all relevant uncertainty (the color of 
the ball drawn from each urn). The rows correspond to 
the acts—in this case, bets yielding 1 unit of utility if 
won and 0 otherwise. The entries in the table are the 
consequences obtained under each act and in each state 
(e.g., if you bet on B, and the state B, & By realizes, you 
obtain a consequence worth 1 utility unit). 


2. Multiple Priors 


A popular reaction to the opening example focuses on 
the precision of the Bayesian representation of beliefs. 
In the unknown urn, there is no evidence to justify a 
particular call on the probability of black on the next 
draw. Yet, by insisting that a precise probability value 
must be assigned to this event, Bayesianism has—the 
thought goes—no way of expressing this ignorance. A 
natural move is thus to allow imprecision in valuations, 
by using sets of probability measures as representations 
of belief. Bayesianism amounts to the special case where 
the agent’s set is a singleton. Such a representation has 
been discussed and defended under names such as credal 
sets in philosophy (Levi, 1986; Joyce, 2010) and impre- 
cise probabilities in statistics (Walley, 1991); in econom- 
ics, one speaks of multiple priors or sets of priors. 

How could or should one decide on the basis of a set 
€ of priors? One of the earliest theories in economics— 
maximin Expected Utility or maximin-EU, developed by 
Gilboa and Schmeidler (1989)*—looks at the worst-case 
expected utility across the set; that is, it considers the 
representation of preferences by 


minpec EUpf. (1) 


The rule is cautious or pessimistic, insofar as it bases pref- 
erences on the worst the act can do, according to the 
probability measures in the set C. As such, it can straight- 
forwardly account for the Ellsberg preferences. (The 
interested reader may verify this using the set C* = {p EA: 
0.3 < p(Bu) < 0.7; p(Bx) = 0.5}.) 


Decision under Uncertainty 


Table 8.4.2 
Elisberg choices 
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B, & B, (Black 
ball drawn from 


B, & W, (Black drawn 
from known urn, white 


both urns) from unknown urn) W,&B Wy, & Wi 
1,,0 (bet on Bx) 1 1 0 0 
1,,0 (bet on B,) 1 0 1 0 
Tw,0 (bet on Wx) 0 0 1 1 
ly,0 (bet on W.) 0 1 0 1 


One purportedly restrictive aspect of this rule is the 
focus on the worst case; an apparently less extreme alter- 
native is the a-maximin-EU or Hurwicz criterion, along the 
lines suggested by Hurwicz (1951; Jaffray, 1989): 


(2) 


where a is a number between 0 and 1. This rule contains 
maximin-EU as a special case (where a= 1) but goes 
beyond the arguably extreme case of total caution by 
taking into consideration both the worst and the best 
case. Other generalizations in this direction allow the a 
to depend more or less strongly on the act f being evalu- 
ated (Ghirardato, Maccheroni, & Marinacci, 2004). 

Another approach focuses on what an agent can say 
“for sure” on the basis of her set of priors. Under a popu- 
lar proposal, a preference for one act over another is 
formed only if all probability measures in the set “agree” 
that the former has higher expected utility. In other 
words, for all acts f, 3, f=g if and only if 


a mMiNpec EUpf + (1- a) Maxpec EUpf, 


EU, f = EUpg, for all p EC. (3) 


This unanimity rule has been defended in economics 
(Bewley, 1986), in statistics under the name maximality 
(Walley, 1991), as well as by several philosophers (S. Brad- 
ley & Steele, 2016).* Note that it does not order certain 
pairs of acts: the reader is invited to check, for instance, 
that it does not order the Ellsberg bets under the set 
of priors C* above. Related rules include Levi’s (1986) 
E-admissibility, which picks out acts that are best accord- 
ing to at least one probability measure in the set. 
Beyond intuitive remarks about their reasonableness, 
how can decision rules such as these be evaluated on 


normative grounds? Two potentially relevant families of 
results have been developed in the field; we shall illus- 
trate them on maximin-EU. 


2.1 Evaluation 1: Implications for Choice 

One way of evaluating the normative credentials of a 
decision rule is in terms of its implications for choice. If it 
leads to unpalatable choices, such as choosing to obtain 
a sure loss, then this provides good reason for skepti- 
cism; if its consequences for choice seem sensible, then 
this may provide arguments in its favor. Any particular 
choice results from the combination of the decision rule 
(such as (1)) and the agent’s attitudes (the set of priors 
and utility function). To focus evaluation on the rule, 
decision theory considers the implications it has no 
matter the agent’s attitudes. For instance, the maximin- 
EU rule generates transitive preferences (see below) for 
any C and u used. The central results in decision under 
uncertainty—representation theorems—fully characterize 
the implications of the decision rule for choice. That 
is, they provide necessary and sufficient conditions on 
preferences—called axioms—for there to be some speci- 
fication of the agent’s attitudes that represents them 
according to the rule. Figure 8.4.1 presents the general 
schema on maximin-EU (for details, see Gilboa, 2009; 
Gilboa & Marinacci, 2013). 

These results help pinpoint the properties of prefer- 
ences that distinguish decision rules from one another. 
For instance, the Bayesian expected utility rule satisfies 
(see chapter 8.1 by Griine-Yanoff and chapter 8.2 by 
Peterson, both in this handbook):* 


if and 
only if 


Preferences > satisfy a (particular) 


set of axioms 


Moreover, the representing C and wu are suitably unique. 


there exists a set of priors C and utility 
function uv such that = is represented 
by (1) calculated with C and w. 


Figure 8.4.1 
Representation theorem. 
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Weak Order (WO): The preference relation is transitive 
(for all f, g, h, f=g and g=h imply f = h) and complete 
(for all f, g, f= 8 or gf). 


Sure Thing Principle (STP): The preference across any 
pair of acts is independent of what the acts yield on 
events where they agree. 


For instance, in table 8.4.2, since 1,,0 and 1,0 agree on 
the event that the balls drawn from the two urns are the 
same color (i.e., the first and fourth states), STP implies 
that preferences over these acts should be independent 
of their consequences on this event. The same goes for 
1y,0 and 1y,0. However, since on the complement event 
(i.e., the event consisting of the second and third states), 
1,,0 coincides with 1,,0 and 1, 0 coincides with 1,,0, STP 
implies that 1,,0 > 1, 0 if and only if 1,0 =1,,0. Hence, the 
Ellsberg preferences (for 1,,0 over 1,,0 and for 1,,0 over 
1,,0) violate STP. 

In terms of choice, the maximin-EU and unanimity 
rules take complementary approaches: the former weak- 
ens STP while retaining WO; the latter drops the com- 
pleteness clause of WO while holding on to STP. Various 
arguments have been proposed, in both the philosophi- 
cal and economic literatures, that violating one or other 
of these axioms leads to unsavory consequences, particu- 
larly in dynamic contexts. They have been used by some 
to argue for, say, dropping WO rather than STP (S. Bradley 
& Steele, 2016; Seidenfeld, 1988). They have also been 
used to criticize any divergence from Bayesianism—and 
hence all of the approaches discussed in this survey—as 
irrational (Hammond, 1988; Al-Najjar & Weinstein, 2009; 
Elga, 2010).° This is currently the main battleground 
between Bayesianism and its critics and deserves a sur- 
vey in itself. Note that even if such arguments are cor- 
rect about the weaknesses of non-Bayesian approaches 
in dynamic contexts, these need to be traded off against 
their strengths in dealing with the role of evidence in 
choice. Some have claimed that the latter outweigh the 
former (Gilboa et al., 2009). 

Apart from dropping STP, maximin-EU retains two 
axioms satisfied by expected utility: 


P4: The preference for a bet on E over a bet on F with 
the same stakes is independent of the stakes. 


Uncertainty Aversion (UA): For any pair of disjoint 
events E, F with 1,0 ~ 10, Y2eu20 = 120. 


Preferences over bets on events—for a bet on B, over a 
bet on B, (table 8.4.2)—reflect agents’ “willingness to bet” 
and are typically considered to be related to their beliefs 
(Savage, 1954). P4 states that the relative willingness 
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to bet on different events is independent of the stakes 
involved—that is, of the consequences of winning or los- 
ing the bet, assuming that these are the same for both 
bets.’ Uncertainty Aversion translates the caution of 
maximin-EU. V2zzur0 “hedges” the uncertainty involved 
in the bets on E and F: the uncertainty is “halved,” inso- 
far as the payoff depends on whether E or F realizes, not 
which one does, but the winnings are halved as well. For 
an agent who dislikes uncertainty, such hedging can only 
be attractive, and hence the hedge is (weakly) preferred 
to the initial bets. This axiom is often taken as the prop- 
erty characterizing uncertainty (or ambiguity) aversion. 
However, it seems that people are not universally ambi- 
guity averse (Wakker, 2010); for instance, in a version of 
the running example with 10 possible colors rather than 
2, there is a tendency to bet on the unknown urn. Were 
such preferences to be deemed reasonable, this would be 
a blow for the rational credentials of maximin-EU. The 
a-maximin-EU rule retains P4 but drops UA and hence 
can accommodate these preferences. 


2.2 Evaluation 2: Beliefs, Tastes, 

and Uncertainty Attitudes 

Another criterion for evaluating a decision rule con- 
cerns its capacity to neatly separate beliefs from tastes 
or values. Consider the following oft-cited criticism of 
maximin-EU. In our running example, you know noth- 
ing about the composition of the unknown umm, so any 
probability measure is possible; the set Co = {p EA: 
0 < p(Bu) <= 1, p(Bx) = 0.5} captures this. However, accord- 
ing to maximin-EU with this set, a bet on the unknown 
urn is considered worse than a bet on black from an urn 
known to contain 1 black ball and 99 white ones. Such 
caution seems extreme. 

This objection relies on two assumptions: first, that 
the set of priors involved in the decision rule represents 
beliefs and, second, that the beliefs should perfectly match 
available information. Concerning the latter assumption, 
the set Cy may reflect the objectively available informa- 
tion, but it does not capture a Bayesian agent’s beliefs, 
for these must be precise; so why should things be dif- 
ferent for non-Bayesians? Indeed, the set of priors is 
often understood to represent the agent’s state of belief, 
which incorporates, but may go beyond, the “objective” 
information (see also R. Bradley, 2017). Indeed, many 
economists subscribe to the revealed-preference para- 
digm (see chapter 8.1 by Griine-Yanoff, this handbook) 
and take the set of priors to be the uniquely determined 
set specified in the appropriate representation theorem 
(figure 8.4.1). 
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Turning to the first assumption, suppose that C; = {p EA: 
0.4 < p(Bu) < 0.6} represents your preferences (according to 
(1)), so you do not hold the preference claimed in the 
objection.* Since this set goes beyond the “objective” infor- 
mation, one can ask why you are using it. Is it because you 
have further information, or an inclination to “believe” in 
the principle of insufficient reason? Or is it instead because 
you are not so cautious as to use Cp but rather have a higher 
tolerance of uncertainty? These two possibilities are radi- 
cally different: the former concerns beliefs (and their for- 
mation), whereas the latter relates to values or tastes for 
bearing uncertainty. Decision theory has developed formal 
tools that can shed light on such questions. 

The central concept is comparative uncertainty aver- 
sion: between two agents, which (if any) is more uncertainty 
averse? The following is a widely accepted behavioral 
definition of the concept: 


Comparative Uncertainty Aversion. Agent 1 is more 
uncertainty averse than agent 2 if, whenever agent 1 
weakly prefers an act f over a sure c, then so does 
agent 2. 


If Ann prefers an uncertain act f to a sure (utility) 
amount c, then she is not so averse to uncertainty as 
to consider bearing the uncertainty involved in f to be 
worse than getting the “non-uncertain” c. If Bob exhibits 
the same preference in all such cases, then he is not that 
averse to uncertainty either, so he is (weakly) less uncer- 
tainty averse than Ann. 

Influenced by the analogy with risk attitude, econo- 
mists have generally considered uncertainty attitude as a 
taste (for bearing uncertainty) and been interested in its 
consequences for, say, investment decisions. In particular, 
they have characterized comparative uncertainty aversion 
in terms of the parameters of decision models, with results 
such as that in figure 8.4.2 for maximin-EU (Ghirardato & 
Marinacci, 2002). It tells us that differences in uncertainty 
attitude correspond to differences in the set of priors. The 
fact that a comparison of tastes or values—as uncertainty 
attitudes are understood to be—translates to a difference 
in the set of priors casts doubt on the claim that this set 
reflects only beliefs. Indeed, the conclusion often drawn 
is that there is no clean interpretation of the set of priors: 
it reflects aspects of both belief and uncertainty attitude 
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(Klibanoff, Marinacci, & Mukerji, 2005). The maximin- 
EU account, it seems, does not have the resources to 
determine whether and to what extent the set C; reflects 
enhanced beliefs or uncertainty tolerance. 

These interpretational subtleties signal a perhaps 
more severe departure from the Bayesian benchmark 
than first imagined. Bayesianism vaunts a clear separa- 
tion of beliefs (captured by the probability measure) and 
tastes (reflected in the utility function); maximin-EU 
apparently does not. To the extent that such a separation 
is desirable, this could bode ill for its normative creden- 
tials. For instance, in public decisions (e.g., concerning 
climate policy), it is customary for one group to supply 
the factual judgments (e.g., climate experts) and another 
to provide the values (e.g., society or its representatives); 
without the separation of beliefs and tastes, this divi- 
sion of labor is compromised. While current arguments 
against value-free science tend to focus on difficulties 
faced in practice (see Douglas, 2009, and chapter 14.2 
by Bueter, this handbook), using sets of priors to report 
uncertainty risks ruling out such a possibility in principle. 
Moreover, the separation allows questions of theoretical 
rationality (e.g., learning) to be treated independently 
of those of practical rationality (e.g., decision; see chap- 
ter 2.2 by Wedgwood, this handbook). But if an agent’s 
set of priors reflects not only the evidence acquired but 
also her uncertainty attitude, then both will play a role 
in the formation of such sets; any theory of belief update 
will thus also have to incorporate value considerations 
related to uncertainty tolerance. To date, work on belief 
updating for sets of priors does not seem to have grap- 
pled with this issue. 


2.3. Across Multiple Prior Rules 

In comparing the various rules for choosing on the basis 
of sets of priors, the belief—taste separation issue seems 
not to favor any particular one but rather concerns the 
multiple prior representation in general: results a la fig- 
ure 8.4.2 for the unanimity rule suggest that it suffers 
from similar problems.’ As concerns implications for 
choice, while some dispute the normative credentials of 
the preferences promoted by rules retaining WO (such 
as maximin-EU), few dispute the preference orderings 
yielded by the unanimity rule: if it recommends against 


if and 
only if 


For agents | and 2 with the same utility 
function, agent 1 is more uncertainty averse 
than agent 2 


agent 2’s set of priors in a subset of 
agent 1’s set of priors. 


Figure 8.4.2 
Comparative Uncertainty Aversion. 
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an act as being worse than another for all probability 
measures, then it would seem like a bad idea to choose 
it. However, as noted previously, the rule may remain 
silent on some comparisons between acts; the issue is what 
to do in these cases. 

Some suggest that there is nothing more to be said 
about rational decision: the unanimity rule provides all 
the guidance there is, and in cases where it is silent, there 
is no more guidance to be had (S. Bradley & Steele, 2016). 
Others invoke “mechanisms” that are specific to such 
cases of “indecision,” for instance, choosing a (contextu- 
ally provided) status quo option, taking a deferral option, 
or choosing at random. Some have suggested “picking” 
a precise probability in the set of priors, or “sharpen- 
ing,” for the purposes of decision (Joyce, 2010). Such 
a procedure needs to be carried out in a coherent way 
across decisions, to avoid agents making chains of deci- 
sions that yield sure losses. Rules such as maximin-EU 
can be thought of as providing principles for “picking” a 
probability: it always chooses a probability measure that 
evaluates the act as badly as possible among those in 
the set. Although the relevant probability measure dif- 
fers according to the act evaluated (the measure used to 
evaluate a bet on white may not be relevant for a bet on 
black), the representation theorem tells us that the rule 
is invulnerable to the aforementioned problems. Indeed, 
each of the rules discussed above satisfying WO can be 
thought of as one way of “complementing” the una- 
nimity approach in situations where it remains silent. 
The possibility of complementing approaches violating 
WO-—seen as a strong rational base—by invoking con- 
siderations such as “security’—which may not enjoy 
the same interpersonally valid rational credentials—has 
been discussed by Levi (1986). Gilboa, Maccheroni, Mar- 
inacci, and Schmeidler (2010) invoke a similar intuition, 
distinguishing “objectively” from “subjectively rational” 
preferences, represented according to the unanimity and 
maximin-EU rules, respectively. They provide axioms 
characterizing the maximin-EU “completion” of given 
unanimity preferences. 


3. Nonadditive Probabilities 


Another reaction to Bayesianism’s apparent difficulties 
in accounting for evidence focuses on the additivity 
of probability functions. A range of proposals, includ- 
ing Dempster-Shafer belief functions, possibility func- 
tions, and Shackle’s degrees of surprise (see chapter 4.7 
by Dubois & Prade, this handbook), employ real-valued 
functions assigning a number in [0, 1] to each event and 
satisfying a mild monotonicity condition: bigger events 
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do not get a lower value. Such (monotonic real-valued set) 
functions are called capacities (other terms used, in various 
literatures, include “fuzzy,” “confidence,” and “plausibil- 
ity measures”). 

A first-pass decision rule involving capacities would 
keep the expected utility formula (table 8.4.1) but with 
capacities in the place of probabilities. It has long been 
known that such a rule has very unattractive behavioral 
properties: any decision maker using it will strictly pre- 
fer some dominated act (i.e., one that does worse in all 
states than another available act; see Quiggin, 1982; Wak- 
ker, 2010). Basically, the main decision rule under uncer- 
tainty using capacities that avoids such problems is the 
“Choquet Expected Utility” or “CEU” rule proposed by 
Schmeidler (1989), which evaluates an act faccording to 


Yi vits: uf (s)) = x PL — Xv], (4) 


where v is the capacity, u is the utility function, and 4; 
are the utility values of consequences of f, in decreasing 
order. Beyond being the main rule suitable for the sorts 
of belief representation mentioned above, it has proved 
popular descriptively (see section 5 below), partly because 
of its implications for choice: the rule does not assume UA 
(although it does satisfy P4), which, as noted, is sometimes 
violated. 

However, UA is satisfied whenever the capacity is con- 
vex, that is, whenever VEU F) + (VEN F) = v(E) + V(F) for 
all events E, F. In this case, the Choquet integral coincides 
with the maximin-EU evaluation using the set of prob- 
ability measures dominating the capacity (called the 
core), that is, {p EA: for all E CS, p(E) = v(E)}. Since sev- 
eral nonadditive probability representations—such as 
Dempster-Shafer belief functions—are convex capacities, 
this means that the decision rule for them is (equivalent 
to) maximin-EU. So the previous remarks also hold 
for them. More generally, results a la figure 8.4.2 indi- 
cate that the capacity in the CEU rule reflects attitude to 
uncertainty (as well as, potentially, beliefs), so the sepa- 
ration issue remains problematic. 


4. Confidence and Other Second-Order 
Representations 


A third approach focuses on a purported multidimen- 
sional character of beliefs: to use Keynesian vocabulary, 
beyond the balance of evidence supporting a probabilistic 
judgment, there is also its weight. The thought that both 
of these “dimensions” are involved in the representation 
of beliefs dates back at least as far as Peirce (1878): “to 
express the proper state of our belief, not one number 
but two are requisite, the first depending on the inferred 
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probability, the second on the amount of knowledge on 
which that probability is based.” Similar distinctions 
have more recently played a prominent role in cognitive 
psychology (Griffin & Tversky, 1992). To capture this 
idea, representations generally employ some structure 
over the probability space A. 

A simple example is a (weak) order over the prob- 
ability space or, equivalently, a nested family of sets of 
probability measures, denoted & (figure 8.4.3). R. Bradley 
(2017) and Hill (2013, 2019) argue that such a structure 
can capture an agent’s confidence in her beliefs. Each set in 
the family represents the beliefs or probability judgments 
held at a given level of confidence: the probability judg- 
ments that hold for all measures in the set are those that 
the agent holds with (at least) the corresponding amount 
of confidence. For instance, an agent represented accord- 
ing to figure 8.4.3 has high confidence in the probabil- 
ity judgment of 0.5 for B, but only low confidence in 
the probability judgment of 0.5 for By. Larger sets in the 
family (for which fewer judgments hold) correspond to 
higher levels of confidence. Conceptually, confidence in 
a probability judgment is related to the evidence under- 
pinning it (R. Bradley, 2017), so this approach relates to 
the aforementioned tradition. Technically, the represen- 
tation is simply the “system of spheres” representation 
from belief revision and conditional logic (see chapter 5.2 
by Rott and chapter 6.1 by Starr, both in this handbook), 
applied over the probability space rather than the state 
space. Belief representations in this spirit have been pro- 
posed by Gardenfors and Sahlin (1982) and Nau (1992), 
although they assume that the second-order structure is 
cardinal: a number is assigned to each probability mea- 
sure. Naturally, every cardinal structure over the proba- 
bility space induces the ordinal structure described above 
(by ordering probability measures according to the value 
assigned). Similar uncertainty representations have also 
been promoted in econometrics (Manski, 2013). 

Like multiple priors (section 2), several decision rules 
operate with this representation of beliefs. A notable 
family encapsulates the arguably reasonable maxim: 
the higher the stakes involved in the decision, the more 


High confidence ———————_y 


Medium confidence 


Low confidence 


{Po: Po(By) = 0.5; Po(B,) = 0.5} 
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confidence is required in a belief for it to play a role. They 
involve a function D assigning to each decision a confi- 
dence level—formally, a set in the nested family = rep- 
resenting the agent’s beliefs—and evaluate acts on the 
basis of the set picked out as appropriate for the decision 
by D. For each multiple-prior decision rule (section 2), 
there is a “confidence version,” involving the same rule 
but allowing the set of priors involved to vary according 
to the decision, and in particular the stakes involved in 
it. For instance, the maximin-EU member of this family 
represents preferences according to 


in EU, f. 5 
eae U,f. (5) 


The main difference for choice with respect to 
maximin-EU lies in P4, which directly clashes with the 
intuition that, for different stakes, different levels of confi- 
dence may be appropriate and hence different beliefs may 
inform the decision, leading to potentially different will- 
ingness to bet. Decision rule (5) satisfies a weakening of 
this axiom, which allows the willingness to bet to change 
with the stakes, in line with the aforementioned maxim. 
Unlike maximin-EU, results a la figure 8.4.2 suggest a clean 
separation between beliefs and values (Hill, 2013): the 
family of sets = represents the beliefs while the function 
D represents the agent’s uncertainty aversion, or taste for 
choosing on the basis of limited confidence. Drawing on 
similar results for the confidence version of the unanim- 
ity rule, Hill (2019) argues that these two points—the mild 
yet motivated divergence from multiple prior models in 
terms of choice implications, and the clean separation of 
beliefs and tastes—are general properties of the approach. 

As noted previously, any cardinal representation of 
confidence in beliefs can be used with this sort of deci- 
sion procedure, by “forgetting” the numbers and using 
only the order. However, other decision rules have been 
developed that make specific use of the cardinal struc- 
ture. Prominent ones are the variational preferences rule 
(Maccheroni, Marinacci, & Rustichini, 2006), which rep- 
resents preferences by 


min(EU, f+¢(p)), (6) 


a {PEA:0.1 <p(B,) < 0.9; p(B, = 0.5} 


{p € A: 0.3 < p(B,) < 0.7; p(By) = 0.5} 


Figure 8.4.3 
Confidence in beliefs. 
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where c is a real-valued function on A, and the confidence 
preferences rule (Chateauneuf & Faro, 2009), 


ll 

Tee gp) eh 
where @ is a function from A to [0, 1]. The former rule 
is motivated by an important literature on robustness 
in macroeconomics: a central model developed there 
(Hansen & Sargent, 2008) is a special case. The latter was 
motivated by and technically related to the literature on 
fuzzy sets: y is a fuzzy set of probability measures. Both 
approaches diverge from maximin-EU by weakening P4, 
without motivating the divergence with a maxim similar 
to that behind (5). However, these models do not cleanly 
separate beliefs and tastes: results a la figure 8.4.2 sug- 
gest that c and mcapture uncertainty attitudes and hence 
cannot be thought of as pure representations of belief. 


5. Probabilities after All 


Finally, some strive to retain the probabilistic represen- 
tation of beliefs. Inspired by work on decision under 
risk, one approach assumes that agents assign a precise 
probability p(E) to each event E but that in choice, these 
probability values are “deformed” by a weighting function 
w, so that the decision weight attached to the event is 
w(p(E)). Since the composition of w and p is a capac- 
ity, this approach, popular in prospect theory (Wakker, 
2010; see chapter 8.3 by Gléckner, this handbook), uses 
the CEU rule (section 3). A different approach reverts to 
a second-order probability u over the space of first-order 
probabilities A. To the extent that this is a second-order 
structure, it can be thought of as related to the confi- 
dence approaches in section 4 (Marinacci, 2015). 

The main challenge for these approaches lies in the 
incompatibility between the Ellsberg preferences and the 
property of probabilistic sophistication, which, roughly, states 
that all the decision-relevant information about an event 
is summarized in the probability assigned to it (Machina & 
Schmeidler, 1992). For instance, in the Ellsberg example, 
if the agent assigns probability 0.5 to both B, and Bu, as is 
often deemed reasonable, then the deformed weight w(0.5) 
assigned to the two events is the same, and the CEU rule 
predicts indifference between the bets. Likewise, a second- 
order probability u generates a “reduced” probability Zpu(p), 
which can be used in the context of the expected utility 
rule; doing so is equivalent to the following evaluation, 
which applies expected utility at both levels: 


EU, f dup) (8) 
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However, as noted at the outset, expected utility cannot 
accommodate the Ellsberg preferences. 

Faced with this situation, a common reply is to treat 
(the probabilities of) different events differently in deci- 
sion according to the type of event, and more specifi- 
cally the source of uncertainty, to which it belongs. The 
Ellsberg known and unknown urns are different sources 
of uncertainty, and so, the idea goes, the same probabil- 
ity assignment with respect to events from the different 
sources can be treated differently in decision. 

Source dependence was introduced in the behavioral 
literature (Tversky & Fox, 1995) and continues to play a 
central role in current developments of one of the most 
important descriptive theories of decision, prospect the- 
ory (Abdellaoui, Baillon, Placido, & Wakker, 2011; Wak- 
ker, 2010; see chapter 8.3 by Gléckner, this handbook). 
Recent versions incorporate source preference by using 
the CEU rule with weighting functions that depend on 
the source of uncertainty, so probabilities about events 
concerning the known urn are weighted differently from 
those concerning the unknown urn. 

For second-order probabilities, the following smooth 
ambiguity decision rule has been proposed (Klibanoff 
et al., 2005):1° 


J (EU, f) dup), (9) 

A 
with qa real-valued function on utility values. The trans- 
formation function ¢ translates the difference in attitude 
to first-order and second-order uncertainty; interpreting 
the former as “physical uncertainty” and the latter as 
“model uncertainty,” it thus reflects different attitudes 
to these two sources of uncertainty (Marinacci, 2015). In 
particular, whenever ¢ is nonlinear, the attitudes differ, 
and the rule can accommodate Ellsberg preferences. This 
is one of the prominent models in the literature admit- 
ting a clean separation of beliefs from uncertainty atti- 
tudes: the second-order probability u can be understood 
as a representation of the agent’s state of belief, whereas 
gy reflects her uncertainty attitude (Klibanoff et al., 2005). 
This separation, combined with the tractability and famil- 
iarity of the largely Bayesian framework, has contributed 
to its increasing popularity in economic modeling. 

While some approaches invoking source dependence, 
particularly in the behavioral literature, have plainly 
descriptive aims, others seem to harbor normative ambi- 
tions (Marinacci, 2015) and hence call for an appraisal 
of their rational credentials. Criticisms have focused on 
their resolution of the challenge of reserving a role for 
weight of evidence in decision (section 1). They insist 
that the Bayesian representation captures all relevant 
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aspects of belief but introduce another element (the gin 
(9)) to account for the role apparently played by weight 
of evidence. Some have suggested that this tension—in 
particular, the use of a parameter representing uncer- 
tainty attitude (q@) to reflect a characteristic of evidence, 
which is a factor pertaining to belief—is damaging for 
the rational credentials of approaches invoking source 
dependence (Hill, 2019). 


Notes 


1. We follow economists in using “decision under uncertainty” 
to refer to cases where the probabilities of the various outcomes 
are not provided, as opposed to “decision under risk,” where 
they are given. 


2. After the notion of uncertainty introduced by Knight (1921). 


3. Similar decision rules have been defended in philosophy 
(Gardenfors & Sahlin, 1982) and developed in robust statistics, 
where they are called “T-maximin” (Berger, 1985). 


4. Different versions vary according to whether the inequali- 
ties are weak (as in (3)) or strict. 


5. Throughout, we retain the standard names for axioms used 
in the literature. For further details, the reader is referred to 
Gilboa (2009) and Gilboa and Marinacci (2013), for example. 


6. While these arguments focus on decision, they are related to 
“dilation” arguments that focus specifically on learning under 
imprecise probabilities (Seidenfeld & Wasserman, 1993; Joyce, 
2010). 


7. This axiom, which was introduced by Savage (1954), whose 
nomenclature we adopt, is related to Gilboa and Schmeidler’s 
C-Independence (1989). 


8. Applying the maximin-EU rule with this set of priors evalu- 
ates a bet on black from the unknown urn using the worst-case 
probability for black—0.4—and this is better than the probabil- 
ity of getting black from an urn with one black ball out of 100. 


9. For a-maximin-EU, despite some indications of belief—taste 
separation (Ghirardato, Maccheroni, & Marinacci, 2004), there 
are reasons for skepticism (Klibanoff, Marinacci, & Mukerji, 
2005). It is complicated by the interdependence between the 
representing set of priors and the a (the uniqueness is weaker 
than in figure 8.4.1), which, without controversial assumptions, 
is unconducive to a clean separation. 


10. For related approaches, see Ergin and Gul (2009), Nau 
(2006), and Seo (2009). 
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8.5 Bounded Rationality: A Vision of Rational Choice in the Real World 
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Summary 


This chapter focuses on the conception of bounded 
rationality introduced by Herbert Simon as an alterna- 
tive to the perfect rationality of the omniscient homo 
economicus, then further developed in psychology and 
economics. Bounded rationality is a principle stating 
that real-world rational cognition is limited by bounds 
on time, computational power, foresight, and knowl- 
edge; it is also a theory of what it means to be rational 
given limited human cognition and an uncertain and 
complex world. We outline the foundations of bounded 
rationality in Simon’s work and explore interpretations 
of bounded rationality in three research programs: in 
economics, the optimization-under-constraints pro- 
gram; in psychology, the heuristics-and-biases program 
and the program on ecological rationality. 


It is time to take account...of the empirical limits on 
human rationality, of its finiteness in comparison with the 
complexities of the world with which it must cope. (Simon, 
1957b, p. 198) 


Broadly stated, the task is to replace the global rational- 
ity of economic man with the kind of rational behavior 
that is compatible with the access to information and the 
computational capacities that are actually possessed by 
organisms ...in the kinds of environments in which such 
organisms exist. (Simon, 1955, p. 99) 


1. Foundations of Bounded Rationality 


1.1 From Metaphors to Models of Choice 

If there was any period in human history during which the 
Enlightenment ideals of rationality and humanity were at 
risk of being shattered completely, it was the 20th century. 
Paradoxically, this century, abounding with the horrors of 
human irrationality and unprecedented self-destruction, 
was also the most fruitful period for advancing both for- 
mal normative theories of rational choice and psycho- 
logically realistic models of reasonable behavior. Herbert 


Simon’s work on bounded rationality was one such 20th- 
century advancement. It established a new requirement 
for theorizing about human choice: the need to under- 
stand how real people, not agents with supreme foresight 
and unlimited computational skills, make decisions when 
faced with limited time and knowledge. The time was ripe 
to comprehend how people achieve their goals in the face 
of a staggering environmental complexity that surpasses 
the limits of the computational resources of any human 
being or any information-processing system. 

Simon introduced the concept of bounded rationality 
in the mid-1950s to describe a new theoretical and empir- 
ical approach to studies of choice and rational behavior. 
First, he defined bounded rationality as a principle that 
emphasizes the limits on computational capacities of 
agents in the real world who have only limited time, 
knowledge, foresight, and cognitive resources: 


The principle of bounded rationality: The capacity of the 
human mind for formulating and solving complex prob- 
lems is very small compared with the size of the problems 
whose solution is required for objectively rational behavior 
in the real world—or even for a reasonable approximation to 
such objective rationality. (Simon, 1957b, p. 198) 


He thereby positioned bounded rationality as an alter- 
native to the concepts of perfect or substantive rational- 
ity rooted in classical rational choice theory that were 
prominent in the neoclassical economics of his day 
(more on this juxtaposition in section 1.2). The idea of 
cognitive limitations and impractical normative stan- 
dards in the real world was already present in his early 
research on decision making within organizations (Simon, 
1947/1957a). Although Simon initially embraced the nor- 
mative standards of rational choice theory, showing that 
real people fall short of these standards,' in later works, 
he progressed toward a positive theory of bounded ratio- 
nality (most notably beginning with Simon, 1955, 1956). 
This extension from principle to positive theory implies 
that bounded rationality is not merely a simplification 
of perfect or substantive rationality but a descriptive and 
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prescriptive model of actual choice behavior in the real 
world. In fact, Simon’s ultimate endeavor was to recon- 
struct and reinvent the “theory of the rational” (Simon, 
1957b, p. 200). On this view, bounded rationality repre- 
sents a new vision of what it means to be rational given 
both the inescapable uncertainty and complexity of 
the world and the limitations of the human cognitive 
system. Both the principle of bounded rationality and 
the vision of bounded rationality as a new descriptive 
and prescriptive theory of rational choice imply that 
understanding (and eventually predicting) the choice 
behavior of real humans is only possible when psycho- 
logical processes are empirically analyzed. This study of 
what Simon called “procedural rationality” ultimately 
amounts to a theory of the efficient computational 
procedures by which agents (individual human beings, 
organizations, or computers) arrive at good solutions to 
the problems they face (Simon, 1976). 

Simon went on to develop his approach to bounded 
rationality across a diverse set of scientific disciplines such 
as economics, cognitive psychology, and artificial intelli- 
gence, adding new levels to his concept over time. One of 
the most important developments concerned his inquiry 
into an adaptive, or ecological, level of rationality. Spe- 
cifically, he stressed that the essence of rational behavior 
consists in how an organism can adapt in order to achieve 
its goals under the constraints of its environment and 
its own cognitive limitations. These two dimensions of 
rational behavior—cognitive and environmental—gave 
rise to the scissors metaphor that encapsulates bounded 
rationality’s theoretical core: “Human rational behavior 
(and the rational behavior of all physical symbol systems) 
is shaped by a scissors whose two blades are the structure 
of task environments and the computational capabilities 
of the actor” (Simon, 1990, p. 7). 

The scissors metaphor was not the only one Simon 
used to portray human choice and choice environments. 
As he recalled in his autobiography, he saw the labyrinth 
as another powerful symbol for decision making. Simon 
(1991) envisioned decision making “in terms of successive 
choices along a branching path” (p. 86) rather than as an 
optimizing process. The first step in his search to turn his 
metaphors of bounded rationality into realistic models of 
rational behavior was to criticize the narrow and unrealistic 
conception of rationality found in neoclassical economics. 


1.2 The Limits of the Olympian Model of Rationality 

Simon (1983) opposed what he called the “Olympian 
model” of rationality (p. 19), which encompasses con- 
cepts such as perfect or substantive rationality, (subjec- 
tive) expected utility theory, and the unfailingly rational 
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homo economicus.” In order to satisfy its normative 
demands, this model requires a decision maker with 
unlimited cognitive resources who can form mental 
models representing, in a collectively exhaustive and 
mutually exclusive way, all future relevant states of the 
world. Simon argued that while rational choice theory 
was worthy of a home in Platonic heaven, it was simply 
out of place in the real world (Simon, 1983, p. 13). In 
“A Behavioral Model of Rational Choice” (Simon, 1955), 
he spelled out the unrealistic assumptions of perfectness 
and logical omniscience that the Olympian approach to 
rationality made about the agent: 


If we examine closely the “classical” concepts of rationality 
outlined above, we see immediately what severe demands 
they make upon the choosing organism. The organism must 
be able to attach definite pay-offs (or at least a definite range 
of pay-offs) to each possible outcome. This, of course, involves 
also the ability to specify the exact nature of outcomes—there 
is no room in the scheme for “unanticipated consequences.” 
The pay-offs must be completely ordered—it must always be 
possible to specify, in a consistent way, that one outcome 
is better than, as good as, or worse than any other. And, if 
the certainty or probabilistic rules are employed, either the 
outcomes of particular alternatives must be known with cer- 
tainty, or at least it must be possible to attach definite prob- 
abilities to outcomes. (pp. 103-104) 


In actual human choice, such demanding require- 
ments can rarely be met. Outside what Savage (1954) 
called “small worlds”—highly simplified environments 
such as monetary gambles—real people cannot live up 
to the ideal of making decisions by specifying all possi- 
ble outcomes, assigning probabilities and values to each, 
and then maximizing the expected payoffs. It would be 
a misrepresentation, however, to assume that people can- 
not live up to this decision-making ideal due to cognitive 
destitution or a mere lack of skill; Simon (1972) credited 
the irreducible uncertainty inherent in any human reck- 
oning about the future as the reason people did not act 
according to the Olympian model. 

Consider firefighters. Like emergency room doctors, 
police officers, and soldiers, firefighters face enormous 
time pressure, high stakes, and inescapable uncertainty. 
They do not have the luxury of mentally generating all 
possible courses of actions, specifying their respective 
outcomes, and then evaluating them. Not only do fire- 
fighters face epistemic uncertainty, but they also face the 
aleatory uncertainty’ inherent in their environment. The 
Olympian model thus seems to be an obscenely lofty 
conception of decision making. Professional experts do 
not make decisions by comparing all possible options 
but rather by generating a single good course of action 
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from the start (Klein, 1998, p. 17). By drawing on a rep- 
ertoire of past situations compiled from real and virtual 
experience, firefighters and their ilk quickly identify a 
plausible course of action and play it out mentally to test 
whether it could work in the current situation. If success 
seems plausible, it will be implemented; otherwise, it will 
be modified or replaced by the next-best option until a 
“good enough” alternative is found. This is an example 
of a fast and intuitive problem-solving performance—one 
for which the expert cannot necessarily describe in detail 
the underlying reasoning processes. Moreover, the key to 
experts’ success lies in the way they approach and ana- 
lyze their environments. Returning to Simon’s scissors, 
one blade—the environment—provides the cues that are 
in turn matched by the second blade—the mind—which 
draws on its resources (e.g., memory) to find a suitable 
decision strategy. On this view, “intuition is nothing more 
and nothing less than recognition” (Simon, 1992, p. 155). 

Next to irreducible uncertainty and time pressure, 
there is another key reason that the Olympian model 
prospers in Platonic heaven but withers in earthly real- 
ity: computational intractability. Consider chess, a game 
Simon studied extensively (Chase & Simon, 1973). 
Chess offers a choice set of about 30 legal moves (a num- 
ber that stays more or less constant until the end of the 
game) and a time horizon of about 40 moves per player 
until one party concedes; 30*° possible sequences (about 
10''*) follow from the original position (see also Shan- 
non, 1950). No human or computer can generate and 
evaluate all these consequences. Even Deep Blue, the 
IBM computer that beat chess world champion Garry 
Kasparov in 1997 and that could examine some 300 
million possible moves per second, had to evaluate posi- 
tions through “a high-performance alpha-beta search 
that expands a vast search tree by using a large number 
of clever heuristics and domain-specific adaptations” 
(Silver et al., 2018, p. 1). Relative to many real-world 
decisions, however, chess is a piece of cake; indeed, “the 
problem space associated with the game of chess is very 
much smaller than the space associated with the game 
of life” (Simon, 1976, p. 72). In many social interactions, 
the rules are not as well defined as they are in chess, 
the set of possible actions is much vaster, there are more 
than two parties involved, and goals are unclear or, even 
worse, in conflict, thus compounding the intractability 
problem with additional complexity. If chess is com- 
putationally intractable and thus beyond the reach of 
Olympian optimization, so are many of the decisions 
people face in the real world. 

In other words, even though the classical under- 
standing of rationality as set out in Olympian models is 
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theoretically coherent and even appealing, it profoundly 
misrepresents the reality of decision making: the set of 
decisions where Platonic heaven and earthly reality 
intersect is very narrow.’ For Simon (1989), the funda- 
mental question for the study of bounded rationality 
was, therefore, “How do human beings reason when the 
conditions for rationality postulated by the model of 
neoclassical economics are not met?” (p. 377). This ques- 
tion has an important implication. Classical concepts of 
rationality not only fail to describe what people actually 
do but also fail to offer procedural advice about how to 
find solutions to many real-world problems. The study 
of boundedly rational decision making, on the other 
hand, is ultimately concerned with both descriptive and 
prescriptive questions—a point to which we return later. 


1.3 Principles of Bounded Rationality 

Firefighters face limits on time, information, and 
certainty—yet they still make decisions, as do emergency 
room doctors and chess players. How do reasonable people 
decide when optimization is out of reach, and how can 
scientists examine this behavior? In order to show how 
Simon answers this question, we distinguish four princi- 
ples underlying his investigation of bounded rationality. 

The first principle calls for a behaviorally informed 
theory of choice, which should take into account and 
specify what conditions govern human behavior in dif- 
ferent real-world situations. In Simon’s (1992) words, 
“The study of the behavior of an adaptive system like the 
human mind is not a logical study of optimization but an 
empirical study of the side conditions that place limits on 
the approach to the optimum” (p. 157). Failing to under- 
stand not just the “design product (the alternative finally 
chosen) but the design process as well” (p. 156) compro- 
mises behavioral science’s ability to explain and predict 
behavior. This postulate was a bone of contention between 
Simon and Milton Friedman, who fiercely defended classi- 
cal economic theory by arguing that “‘complete realism’ is 
clearly unattainable, and the question whether a theory is 
realistic ‘enough’ can be settled only by seeing whether it 
yields predictions that are good enough for the purpose at 
hand” (Friedman, 1953, p. 41). Simon, however, insisted 
that a realistic account of the cognitive processes was 
required to successfully predict human behavior (see also 
Simon, 1976, on procedural rationality). 

Asecond principle is that realistic models will be models 
of “approximate methods” (Simon, 1990, p. 6). “Approxi- 
mate” here should not be interpreted to mean that these 
methods will produce inferior solutions or lack cognitive 
sophistication. Simple methods, for instance, can exploit 
evolved cognitive, visual, and motoric capacities that can 
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be complex and demanding but are nevertheless easy for 
the mind to execute, such as pattern recognition, emo- 
tions (see Hanoch, 2002), tracking motion in space, and 
the ability to adaptively forget (Schooler & Hertwig, 2005). 
By exploiting the mind’s sophisticated capacities, approxi- 
mate methods can remain computationally slim and oper- 
ate under constraints on time and information. 

A third principle of bounded rationality is that when 
optimization is out of reach, people satisfice—they look 
for good-enough answers. Satisficing does not mean 
merely taking a good-enough alternative and giving up 
on the best possible alternative. Rather, satisficing is “the 
process of finding alternatives by heuristic search with 
the use of a stop rule based upon adjustable aspirations” 
(Simon, 1982, p. 323). In natural situations, alternatives 
are often encountered sequentially (e.g., potential mat- 
ing partners; Miller & Todd, 1998), and the number of 
available alternatives is almost always too large to be 
exhaustively explored; furthermore, alternatives may 
vanish if not chosen immediately. Under these circum- 
stances, a limited search is implemented and the point 
at which a search will be terminated is determined 
by an aspiration level—“a goal variable that must be 
reached or surpassed by a satisfactory decision alterna- 
tive” (Selten, 2001, pp. 13-14). Aspiration levels can be 
adjusted depending on one’s success in finding satisfy- 
ing options. Note that not every boundedly rational 
behavior requires satisficing. Rather, a range of heuristic 
processes can be employed to reach good decisions in 
both strategic games and games against nature (Hertwig 
& Herzog, 2009; see also section 2.3 of this chapter). 

A final principle of the empirical study of bounded 
rationality requires taking into account the properties of 
an organism’s environment. In The Sciences of the Artificial 
(1969/1996), Simon argued that an organism’s behavior 
results not merely from its inherent characteristics but also 
from the structures of its surroundings. Taking the path 
of an ant on a beach as an example, Simon claimed that 
an “ant, viewed as a behavior system, is quite simple. The 
apparent complexity of its behavior over time is largely a 
reflection of the complexity of the environment” (p. 52). 
An ant’s environment may be an obstacle course, forcing 
the ant to repeatedly change direction around pebbles, 
rocks, and puddles (figure 8.5.1). An observer who saw 
only the ant’s laborious path, without the properties of 
the environment, may be tempted to attribute substan- 
tial complexity to the cognitive mechanisms underlying 
the ant’s behavior. Yet, if the ant’s behavior is understood 
as the result of its interaction with its environment, the 
explanation may be much more mundane. The mecha- 
nisms producing the complex path may be simple rules 
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such as, “If the left vision field sensors for obstacles lights 
up, turn to the right, and vice versa.” Even though actual 
ants might be guided by a multitude of other factors as 
well, this example reveals that the study of bounded 
rationality must include the study of the structural prop- 
erties of the surroundings in which organisms, whether 
ants or humans, make decisions. Explaining complex 
behaviors exclusively in cognitive terms risks misattribut- 
ing—or worse, profoundly misconstruing—the causes of 
the complexity. 

Having clarified the basic principles behind Simon’s 
view, we now turn to various research programs in psy- 
chology and economics that claim to have further devel- 
oped the theory of bounded rationality. As we will see, 
these programs are quite diverse and give varying weight 
to the theoretical and methodological notions and princi- 
ples reviewed above. 


2. Interpretations of Bounded Rationality 


There are at least three main research programs aiming 
to interpret bounded rationality and develop it further (see 
also Gigerenzer & Selten, 2001): the economic research 
program of optimization under constraints, the approach in 
psychology known as the heuristics-and-biases program that 
led to the foundation of behavioral economics (and behav- 
ioral law), and a further program in psychology known as 
ecological rationality or the study of fast-and-frugal heuristics. 


2.1. Optimization under Constraints 
Simon’s candid and fundamental criticism of neoclas- 


sical economics and its view of global and omniscient 


Figure 8.5.1 

An ant walking on a beach illustrates that a behavior (here, the 
path the ant takes) that might appear complex to an external 
observer is actually a function of the interaction between the 
organism and its environment. 
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rationality was generally not met with enthusiasm by 
economists. However, in an attempt to reconcile neo- 
classical economics with behavioral and cognitive limits, 
George Stigler, a leading figure of the Chicago school of 
economics, proposed a notion often described as “opti- 
mization under constraints” (Stigler, 1961).° As a former 
student of Frank Knight, Stigler (1961) was mindful of the 
role of uncertainty in human choice, stating that “our 
understanding of economic life will be incomplete if we 
do not systematically take account of the cold winds of 
ignorance” (p. 224). 

In order to incorporate ignorance—or, to use a less 
pejorative term, incomplete knowledge—in the process of 
choice, Stigler (1961) focused on one key idea of bounded 
rationality: the concept of limited search (and, by exten- 
sion, the respect for real decision makers’ finite resources). 
The method for determining when to stop searching is 
interpreted in terms of a cost-benefit analysis. On this view, 
conditions of limited search and less-than-perfect informa- 
tion appear to easily square with models of optimization. 
Specifically, the models in this class assume that a stop- 
ping rule optimizes with respect to the relevant currency 
(e.g., time, computation, money). Thus, when the cost of 
searching for new alternatives or other pieces of informa- 
tion exceeds the benefits of further exploration, the search 
will be terminated. For example, when looking for a new 
car, a buyer explores the market until the expected mar- 
ginal costs (i.e., time and effort) of looking for new options 
exceed the expected marginal benefits (i.e., finding a car 
that more closely meets the buyer’s preferences). Although 
this conception of psychologically realistic decision mak- 
ing apparently reconciles bounded rationality with opti- 
mization (see also Sargent, 1993), the cost-benefit analysis 
imposes the “burden of estimating the expected marginal 
return of search and the opportunity cost” (Simon, 1982, 
p. 296). Indeed, optimization under constraints requires 
the same amount of information and even more knowl- 
edge and computation (e.g., determining the costs of con- 
tinuing search, such as search’s opportunity costs; Vriend, 
1996) compared to neoclassical optimization models. 
Moreover, as Simon (1979) noted, it “poured the search 
theory back into the old bottle of utility maximization” 
(p. 503) without paying due attention to a key element of 
the theory of bounded rationality, namely, that choice can 
be made with incomplete information and without carry- 
ing out optimization procedures. 


2.2 Heuristics and Biases 

The theory of bounded rationality is meant to be a 
behaviorally and psychologically grounded approach to 
decision making. One of the research programs that has 
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contributed significantly to developing this approach is 
the work instigated by Amos Tversky and Daniel Kahne- 
man. Their heuristics-and-biases approach has mapped 
the impact of cognitive limitations on people’s judg- 
ments and decisions and documented a large catalog of 
systematic deviations from norms of rationality, drawn 
from probability theory, statistics, and axioms of ratio- 
nal choice (Kahneman, Slovic, & Tversky, 1982; Tversky 
& Kahneman, 1974). Kahneman interpreted bounded 
rationality in terms of behavior that diverges from such 
norms and, by extension, optimality. In this spirit, he 
claimed, “Our research attempted to obtain a map of 
bounded rationality, by exploring the systematic biases 
that separate the beliefs that people have and the choices 
they make from the optimal beliefs and choices assumed 
in rational-agent models” (Kahneman, 2003, p. 1449). 

This statement both highlights the key normative 
assumption of the heuristics-and-biases program and 
outlines the contours of its map of bounded rationality. 
Rational-agent models (Olympian models, according to 
Simon) as well as their axioms and assumptions (e.g., 
Bayesian probability updating) are explicitly acknowl- 
edged as the benchmarks against which people’s judg- 
ments should be compared. This means that the decision 
problems in question are ones in which an optimal or 
normative solution exists. By extension, bounded ratio- 
nality is not so much a map of human choice in the world 
of the unknown and the uncertain as it is an empirical 
collection of decisional deficiencies in problems with 
assumed normative solutions. This is also evident in 
prospect theory, the most influential descriptive alterna- 
tive choice theory to expected utility theory (Kahneman 
& Tversky, 1979; Tversky & Kahneman, 1992). Indeed, as 
Kahneman emphasized, their aim was to “assemble the 
minimal set of modifications of expected utility theory 
that would provide a descriptive account of . . . choices 
between simple monetary gambles” (Kahneman, 2000, 
p. x). The theory’s goal was to explain a wide range of 
violations of axioms and predictions of expected utility 
theory, thus accepting the normative force of the the- 
ory. Prospect theory enriches the expected utility frame- 
work by introducing a few psychological concepts (e.g., 
nonlinear probability weighting, reference points, loss 
aversion; for a detailed account of prospect theory, see 
chapter 8.3 by Glockner, this handbook). Yet its scope, at 
least in its original form, is confined to the world of risk 
(Savage’s small worlds) with known possible outcomes 
and without the possibility of surprises. As Simon repeat- 
edly stressed, these are not the conditions under which 
individuals and organizations make decisions most of 
the time. 


510 


Outside of the domain of risky choice, the heuristics- 
and-biases program has invoked a small set of heuristics, 
or mental shortcuts, to explain the myriad errors—also 
called “cognitive illusions”—that people commit in their 
inferences and decisions. Specifically, the program’s 
argument is that “people rely on a limited number of 
heuristic principles which reduce the complex task of 
assessing probabilities and predicting values to simpler 
judgmental operations. In general these heuristics are 
quite useful, but sometimes they lead to severe and sys- 
tematic errors” (Tversky & Kahneman, 1974, p. 1124). 

Three key heuristics—representativeness, availability, 
and anchoring and adjustment—have been proposed to 
account for a wide range of cognitive illusions such as 
the base-rate fallacy, the overconfidence bias, the con- 
junction fallacy, the hot-hand fallacy, and the failure to 
appreciate regression to the mean or the law of large num- 
bers. Two theoretical premises buttress these heuristics. 
One is that of attribute substitution, such as similarity- 
for-probability substitution, which is the mechanism 
behind the representativeness heuristic (Kahneman & 
Frederick, 2002). It describes the idea that a heuristic 
evaluates a target attribute of a judgment object (e.g., 
the likelihood that X is a Y or a Z) by substituting it 
with a property of the object that comes more readily to 
mind (e.g., the degree to which X resembles a Y or a Z). 
This occurs when a person who learns that Linda is an 
outspoken woman concerned with social justice judges 
that it is more likely that Linda is a “feminist bank 
teller” than—as is necessarily at least as likely—a “bank 
teller,” because they believe Linda’s characteristics are 
representative of a feminist (Tversky & Kahneman, 
1983). Error-prone attribute substitution occurs when 
three conditions are satisfied: first, the target attribute 
(e.g., probability) is cognitively relatively inaccessible; 
second, the substituting attribute (e.g., similarity) is 
highly accessible; and, third, the mind’s “controller” 
does not reject the substitution. 

This controlling entity relates to the second theoreti- 
cal premise key to Kahneman’s (2011) view of bounded 
rationality—namely, that two, admittedly fictitious, cog- 
nitive systems make up the human mind. System 1 is 
automatic, effortless, and associative. Reasoning errors are 
typically the product of this speedy system and its pro- 
pensity to propose the solution that most readily comes 
to mind. The awareness that an error may have occurred 
requires the more reflective and effortful System 2. This 
system acts like a teacher identifying a student’s mistake 
and rectifying the process that produced it. But System 2 
also tires easily. Too often, instead of analyzing the pro- 
posals of System 1, System 2 is content with the solutions 
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offered, thus leaving decision makers unaware that they 
are about to commit an error. 

The heuristics-and-biases interpretation of bounded 
rationality in terms of a map of the systematic deviations 
between people’s behavior and norms of rationality has 
been highly influential within psychology and its neigh- 
boring fields and has even prompted new fields of research 
such as behavioral economics (e.g., Thaler, 2016) and 
behavioral law (e.g., Sunstein, 2000). However, it has also 
provoked a number of critical objections. One is that the 
standards against which human reasoning is compared in 
this program are narrow and predominantly content-free 
coherence norms (Arkes, Gigerenzer, & Hertwig, 2016). 
Another is that the identified cognitive illusions are by no 
means as impervious to change (e.g., via different formats 
for representing probabilistic information; see, e.g., Hof- 
frage, Lindsey, Hertwig, & Gigerenzer, 2000) as has often 
been claimed. Finally, it has been demonstrated that at 
least some behaviors seemingly based on cognitive illu- 
sions are not best understood as manifestations of human 
irrationality; instead, they may be the product of evolved 
and adaptive decision rules. These rules, however, are not 
adapted to experimentalists’ simplified environments and 
so are wrongly interpreted as irrational behavior (Fawcett 
et al., 2014; see also chapter 10.6 by Cosmides & Tooby, 
this handbook, for an evolutionary perspective on human 
reasoning). 

Interestingly, this is a development already predicted 
by Simon (1957b), who wrote, “I believe that the return 
swing of the pendulum will begin, that we will begin to 
interpret as rational and reasonable many facets of human 
behavior that we now explain in terms of affect” (p. 200). 


2.3 Ecological Rationality 

The third influential interpretation of bounded rational- 
ity is ecological rationality and the study of fast-and-frugal 
heuristics.° This approach aims to understand boundedly 
rational decision mechanisms based on how they match 
the statistical structures of choice environments. Following 
Simon’s emphasis on understanding and modeling rational 
behavior as shaped by both cognitive and environmental 
structures, research on ecological rationality explores the 
adaptive toolbox of simple heuristics that human minds 
have developed through individual, cultural, or evolution- 
ary learning (see Gigerenzer, Todd, & ABC Research Group, 
1999; Hertwig, Hoffrage, & ABC Research Group, 2013; 
Hertwig, Pleskac, Pachur, & the Center for Adaptive Ratio- 
nality, 2019; Todd, Gigerenzer, & ABC Research Group, 
2012). Simple heuristics, understood as the boundedly 
rational mind’s main tools in games against nature and 
social games (Hertwig & Herzog, 2009), are defined as 
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follows: “A heuristic is a strategy that ignores part of the 
information, with the goal of making decisions more 
quickly, frugally, and/or accurately than more complex 
methods” (Gigerenzer & Gaissmaier, 2011, p. 454). 

Proposing a new “theory of the rational” (Simon, 
1957b, p. 200), ecological rationality strives to model 
rational behavior in terms of correspondence norms 
(Hammond, 2000). Correspondence norms represent a 
measure of cognitive success in achieving one’s goals in 
the world—a consequentialist interpretation of rational- 
ity in cognition (Kozyreva & Hertwig, 2021; Schurz & 
Hertwig, 2019; see also chapter 1.3 by Schurz, this hand- 
book). Depending on the environment, cognitive success 
could be measured in terms of criteria such as accuracy, 
speed, frugality, robustness, or accountability. An impor- 
tant methodological principle in research on ecological 
rationality is comparatively testing heuristic strategies 
against informationally and computationally complex 
strategies. Ideally, the success of each strategy should be 
measured against that of others and across a reference 
class of environments (for examples of such compara- 
tive tests, see Gigerenzer & Brighton, 2009; Spiliopoulos 
& Hertwig, 2020). An important insight stemming from 
tests involving a set of strategies and environments is 
that heuristics do not result in good or bad performance 
in and of themselves; a heuristic’s performance is rela- 
tive to the structure of the environment in which it is 
employed. If a heuristic is able to exploit the structure 
at hand, the heuristic can be surprisingly accurate. Simi- 
larly, it can yield dismal performance when facing envi- 
ronment structures that do not match its architecture 
or its assumptions about the environment. Just as the 
ant’s complex path cannot be understood apart from the 
environment in which it unfolded, cognitive tools and 
their ecological rationality cannot be divorced from the 
environmental structures they may exploit. 

Research on ecological rationality has produced 
three key discoveries pertaining to the understanding of 
bounded rationality. First, bounded rationality does not 
mean that simplicity in search, information integration, 
and decision rules will inevitably result in second-rate 
performance. In fact, less is often more. Second, bounded 
rationality requires an ongoing examination of the envi- 
ronmental structures that support heuristics’ perfor- 
mance. To “describe, predict and explain the behavior of 
a system of bounded rationality, we must both construct 
a theory of the system’s processes and describe the envi- 
ronments to which it is adapting” (Simon, 1990, p. 7). 
One could go even further. Rather than cataloging piece- 
meal descriptions of environments, one could harness 
existing theories of environments that shed light on the 


511 


factors and dynamics that amplify, attenuate, or even 
wipe out the environmental structure in question. Based 
on such a theory, one would then examine how the envi- 
ronmental processes interact with theories of the system’s 
process—for an illustration of this approach, see an analy- 
sis of the risk-reward structure using the foraging theory 
of ideal free distribution in Pleskac, Hertwig, Leuker, and 
Conradt (2019). Third, heuristics can enable good perfor- 
mance in a wide range of domains (e.g., judgment, choice, 
inference, classification) in which people face uncertainty, 
lack of information, and time pressure. 

In briefly reviewing the last discovery relating to the 
often surprising success of heuristics, we start with the 
question of why people recruit boundedly rational heuris- 
tics to begin with. The classical explanation is that people 
resort to heuristics in a kind of compromise, saving cogni- 
tive effort at the cost of accuracy (Shah & Oppenheimer, 
2008). Humans and other animals rely on heuristics 
because searching for and processing information can be 
taxing, and heuristics offer relief by trading reduced accu- 
racy for faster, more frugal cognition. This accuracy-—effort 
trade-off—variously conceptualized as a rational and ben- 
eficial trade-off (Payne, Bettman, & Johnson, 1993) or as 
one that is constrained by capacity limitations and paid 
for in terms of reasoning mistakes (the heuristics-and- 
biases view)—has been touted as a potentially universal 
law of cognition. But there is an alternative explanation 
for why people employ heuristics: less computation and 
less information can result in performance that is as 
good as, or even better than, that of computationally and 
informationally complex strategies. In a wide range of 
studies, often involving computer simulations and ana- 
lytical work, the ecological rationality program has dem- 
onstrated numerous instances of such less-is-more effects 
(Gigerenzer, Hertwig, & Pachur, 2011; Spiliopoulos & Her- 
twig, 2020). 

One way to understand these effects is through the 
distinction between prediction and fitting. Prediction 
occurs when the data have not yet been observed and 
a model, whether heuristic or complex in nature, with 
fixed parameter values is employed to predict future 
events. Fitting, in contrast, occurs when the data have 
already been observed and the parameters of a model 
are estimated in order to maximize the fit between the 
data and the model’s behavior. Generally, the more free 
parameters a model has, the better its fit. This rule, how- 
ever, does not hold for predictions. When parameters 
need to be estimated from small or unreliable samples, 
the function between predictive accuracy and model 
flexibility (e.g., the number of free parameters) is typi- 
cally inversely U-shaped. This means that models that 
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are too simple (e.g., have too few parameters) or too 
complex can fail in prediction (Pitt, Myung, & Zhang, 
2002). Heuristics—at least some—can occupy the sweet 
spot between too little and too much complexity. 

The distinction between prediction and fitting is related 
to another theoretical construct relevant to understand- 
ing why heuristics can perform so well: the bias—variance 
framework (exploited in machine learning; Geman, Bie- 
nenstock, & Doursat, 1992). Heuristics can succeed because 
they smartly trade off two components of prediction error: 
bias (how well, on average, the model can agree with the 
ground truth) and variance (the variation around this aver- 
age). Heuristics tend to have higher bias but lower variance 
than more complex models (with more adjustable par- 
ameters), which explains why heuristics perform relatively 
better when making predictions from small environmen- 
tal samples (Gigerenzer & Brighton, 2009; Katsikopou- 
los, Schooler, & Hertwig, 2010). Variance is a substantial 
source of error when information about the environment 
is sparse, thus exposing more complex models to the risk 
of overfitting by virtue of the flexibility granted by their 
parameters. Heuristics, however, are less flexible and thus 
less likely to overfit, which gives them the chance to out- 
perform more complex models when knowledge about the 
environment is incomplete and uncertainty is high. 

Still another approach to understanding heuristics’ 
ecological rationality is in terms of the match between 
environmental structures (i.e., statistical properties that 
reflect patterns of information distribution in the given 
ecology) and cognitive strategies (Hogarth & Karelaia, 
2006, 2007). For instance, one notable property of an 
environment is the presence of a noncompensatory cue. 
This is an attribute that has a much higher correlation 
with the ground truth than all other attributes com- 
bined, which means it cannot be outweighed (compen- 
sated) by any combination of less valid cues or attributes 
(for details and other environmental structures, see 
Simsek, 2013). Noncompensatory cues can be exploited 
by lexicographic heuristics such as Take-the-Best (Giger- 
enzer & Goldstein, 1996) and LEX (Payne et al., 1993) 
that order attributes according to importance and pro- 
cess them sequentially. 

Finally, another significant contribution of ecologi- 
cal rationality explores the ways in which heuristics 
can be engineered to foster better decisions in uncer- 
tain circumstances. As we have seen, the heuristics-and- 
biases program has focused on a small set of heuristics 
and refrained from proposing heuristics to help people 
make better decisions. This is consistent with the view 
that heuristics cannot compete with, let alone outper- 
form, more complex models. In contrast, the ecological 
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rationality program has suggested and inspired new 
heuristics (often in terms of fast-and-frugal trees; Mar- 
tignon, Katsikopoulos, & Woike, 2008) for a wide range 
of professional uses, including geographic profiling, pre- 
scribing antibiotics, and allocating marketing resources 
(see Gigerenzer et al., 2011). This ongoing work suggests 
that the study of boundedly rational heuristics not only 
examines which heuristics people use but also addresses 
profound prescriptive concerns, thereby providing the 
basis for boosting people’s competences by giving them 
smart and easy-to-use tools and decision aids (Hertwig 
& Griine-Yanoff, 2017; the boosting approach to public 
policy making is an alternative to the nudging approach 
that is rooted in the heuristics-and-biases program; Thaler 
& Sunstein, 2008). 


3. Conclusion 


Having already inspired groundbreaking investigations 
into a map of cognitive illusions and an adaptive tool- 
box of ecologically rational heuristics, the exploration 
of bounded rationality has nevertheless in many regards 
only just begun. Great challenges are ahead, including 
constructing an encompassing theory-based taxonomy 
of environmental structures that decision makers perceive 
and the heuristics that exploit them. Another largely 
neglected research topic is how simple heuristics, by 
interacting with environment structures as well as with 
the heuristics of other decision makers, give rise to com- 
plexity in the environment (see, e.g., Hertwig, Davis, & 
Sulloway, 2002; for a related concern, see Schelling, 2006). 
In the years to come, the study of bounded rationality 
should further explore and detail the content of the adap- 
tive toolbox beyond simple heuristics, investigating other 
cognitive tools that people—both as individuals and in 
groups—use to handle uncertainty (see Hertwig et al., 
2019). Perhaps the most ambitious challenge of all is to 
uncover the extent to which integrating the sciences and 
methodologies of heuristics, search, and learning (Her- 
twig, 2015; Lejarraga & Hertwig, in press) can offer a uni- 
fied framework for the study of the boundedly rational 
mind. In a future of vast complexity, informational afflu- 
ence, and potential for surprise, the need for a realistic 
vision of rational choice will only grow—and with it the 
scientific exploration of bounded rationality. 
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Notes 


1. In the second edition of Administrative Behavior, Simon 
(1947/1957) wrote, “Administrative theory is peculiarly the 
theory of intended and bounded rationality—of the behavior of 
human beings who satisfice because they have not the wits to 
maximize” (p. xxiv). He later asserted, “In Administrative Behav- 
ior, bounded rationality is largely characterized as a residual 
category—rationality is bounded when it falls short of omni- 
science” (Simon, 1979, p. 502). 


2. Simon employed several terms throughout his writings 


mu 


to define this theory, including “global rationality,” “perfect 


mou 


rationality,” “objective rationality,” “substantive rationality,” 
“full rationality,” “homo economicus,” and “Olympian model 
of rationality.” The target of these descriptions was the domi- 
nant classical rational choice theory, including, most notably, 
expected utility and subjective expected utility theories. In ratio- 
nal choice theory, norms of coherence and axioms of expected 
utility theory (e.g., transitivity, completeness) are taken as 
benchmarks of rational decision making. This approach, some- 
times described as the standard picture of rationality (Stein, 
1996), remains the most commonly used normative approach 


to rationality in decision sciences. 


3. The distinction between aleatory and epistemic uncertainty 
(or objective and subjective uncertainty) stems from the dual- 
ity inherent in the modern concept of probability, which 
encompasses both epistemic probability (subjective degrees of 
belief) and aleatory probability (stable frequencies displayed by 
chance devices; Hacking, 1975/2006). In a similar vein, epis- 
temic uncertainty refers to incomplete knowledge or informa- 
tion, whereas aleatory uncertainty stems from the statistical 
properties of the environment, which exist independently of a 
person’s knowledge (Kozyreva & Hertwig, 2019). 


4. In his Nobel Prize lecture, Simon (1979) stated, “A strong posi- 
tive case for replacing the classical theory by a model of bounded 
rationality begins to emerge when we examine situations involv- 
ing decision making under uncertainty and imperfect competi- 
tion. These situations the classical theory was never designed to 
handle, and has never handled satisfactorily” (p. 497). 


5. The idea of optimization under constraints was originally 
outlined by Good in 1952 when he observed that agents in the 
real world seek to minimize costs involved in obtaining infor- 
mation and making decisions (in terms of both time and effort; 
see also Wheeler, 2018/2019). 


6. This use of the term “ecological rationality” is not to be 
confused with the terminologically similar but conceptually 
different concept developed by Vernon Smith (2003). 
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8.6 Reasoning, Rationality, and Metacognition 


Valerie A. Thompson, Shira Elqayam, and Rakefet Ackerman 


Summary 


In this chapter, we explore how the study of metacog- 
nition advances our understanding of human rational- 
ity. “Metacognition” refers to the processes by which we 
monitor and control our ongoing cognitive activities. 
These monitoring processes manifest as feelings on the 
continuum between certainty and uncertainty, which 
then are the stimulus for control processes (e.g., a feel- 
ing of uncertainty in an answer is a stimulus to rethink 
that answer, search online). Because feelings of (un)cer- 
tainty are based on heuristic cues, such as the ease with 
which an answer comes to mind, they can mislead our 
control processes and produce unwarranted levels of 
confidence in our decisions. We also argue that metacog- 
nitive principles can help us to understand why people 
are not the cognitive misers that they are assumed to be 
and why they continue to waste cognitive resources on 
difficult or unsolvable problems. 


1. Reasoning, Rationality, and Metacognition 


Here, we explore how understanding metacognitive 
processes can shed light on issues of rationality. At first 
glance, the connection between metacognition and ratio- 
nality may seem straightforward. Metacognition, after all, 
is typically understood to be “thinking about one’s think- 
ing,” a process that seems integral to achieving rational 
decisions.’ However, most researchers are less concerned 
with this self-reflective stance and more concerned with 
the ubiquitous processes by which we monitor and con- 
trol our ongoing cognitions. By “monitoring,” we mean 
the processes that keep track of how well we are doing; 
control processes include changing strategies, maintain- 
ing course, or giving up. 

Monitoring and control again sound like high-level, 
reflective processes, and there is no doubt that they can 
be. In most ordinary instances, however, they operate 
passively (Koriat, 2007), in much the same way as a ther- 
mostat passively monitors the temperature of a room. 


The associated control process would be to send a sig- 
nal to the furnace to start producing warm air when the 
temperature drops below a certain threshold (Thomp- 
son, 2016). Examples include the following: 


1) You meet someone whose face you recognize but 
whose name you are not certain of. You decide not 
to address her by name. 


2) You order a desk from a furniture store and it arrives, 
unassembled. Your first thought is “I can’t put that 
together.” You then look at the instructions, recon- 
sider, and begin assembling. Halfway through, you 
realize that your first judgment was right and call a 
friend to help. 


3) You are working on writing a manuscript and are 
dissatisfied with the progress. Even though the words 
accurately convey your thoughts, you think that it 
does not have that certain ring that will convince 
readers about the points you wish to make. You 
decide to take a break and make a fresh start in the 
morning. 


In each of these examples, a set of cognitive pro- 
cesses is unfolding, involving memory retrieval, solving a 
problem, writing a manuscript, and a second set of pro- 
cesses layered on top, which we refer to as “monitoring 
and control processes.” The monitoring processes are 
assumed to be continuous and, as above, to manifest pri- 
marily as a sense of (un)certainty about how well things 
are proceeding. In this chapter, we will primarily be con- 
cerned with monitoring and control processes associated 
with reasoning, problem solving, and decision making, 
an area of study termed “Meta-Reasoning” (Ackerman & 
Thompson, 2015, 2017). 

The meta-reasoning framework posits many different 
kinds of monitoring judgments. A judgment of solvability 
is an initial assessment of whether a problem is solvable 
and whether one personally is capable of solving it, as 
the self-assembly example in number 2 illustrates. The 
feeling of rightness (FOR) accompanies an initial, intui- 
tive answer to a problem, as illustrated by example 1. 
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Examples 2 and 3 illustrate the ongoing nature of moni- 
toring and the assessment of intermediate confidence. 
At the end of a process, one is left with a final sense of 
confidence or feeling of error that one has successfully 
or unsuccessfully completed the task at hand. 

The reason these monitoring judgments matter is 
their link to control processes: feelings of certainty and 
uncertainty are the arbiters of action (Thompson, 2016). 
A feeling of certainty is a signal that all is well and to carry 
on with the present course of action. Uncertainty signals 
the need to reassess. For example, reasoning problems 
that cue a strong FOR to an initial answer receive less 
reanalysis than problems that cue a low FOR, whereas a 
low FOR is a cue to look for alternative answers (Thomp- 
son, Prowse Turner, & Pennycook, 2011); similarly, low 
intermediate confidence is a signal to give up (Ackerman, 
2014) or seek additional resources, as in the self-assembly 
example, and initial judgments of solvability (example 
2) predict persistence in problem solving (Lauterman & 
Ackerman, 2019). 

The trouble is that these monitoring judgments are 
usually made implicitly, on the basis of heuristic cues 
(Koriat, 1997). Heuristic cues can be based on general self- 
perceptions (e.g., “I am no good at DIY projects”), global 
information about the task (e.g., “These instructions seem 
particularly opaque”), and characteristics of individual 
items (see Ackerman, 2019, for a review). One item-based 
cue is fluency, or the ease with which an answer comes 
to mind. Fluency is a cue to confidence and FOR (Acker- 
man & Zalmanov, 2012; Thompson, Evans, & Campbell, 
2013), regardless of accuracy. That is, answers that come 
to mind fluently are experienced positively (Thompson & 
Morsanyi, 2012) and lead to a sense of certainty, as does 
familiarity (Reder & Ritter, 1992). 

The implications for human rationality are clear: 
because the monitoring processes can send misleading 
signals, control processes may not operate in an opti- 
mal manner. That is, one might believe that a problem 
is solvable, leading one to persist in trying to solve it, 
even when it is not (Lauterman & Ackerman, 2019). 
Similarly, one might have a strong FOR in a suboptimal 
answer and thus fail to think further; conversely, one 
might have a weak FOR in a good solution and thus 
spend unnecessary time thinking it over (Thompson 
et al., 2011; Thompson & Johnson, 2014). Indeed, one 
of the reasons that people commit errors on traditional 
heuristics-and-biases tasks is that the first answer tends 
to come to mind fluently and is therefore held with 
high confidence (Thompson et al., 2013). As a result, 
one may fail to reflect further on the answer and give 
an answer based on beliefs, stereotypes, or familiarity, 
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when the correct answer requires one to think logically 
or probabilistically. 

In this chapter, we consider the advantages and draw- 
backs of the proper allocation of deliberate thinking and 
the role of normative theories, such as logic and proba- 
bility, as markers of rationality. As stated earlier, the goal 
of this chapter is to consider how understanding meta- 
cognitive processes can advance our understanding of 
human rationality. In doing so, we examine how meta- 
reasoning and metacognition shed new, and sometimes 
surprising, light on rationality, when considered from a 
variety of perspectives. In addition to normative correct- 
ness, we consider issues related to pragmatic rationality, 
bounded rationality, and epistemic rationality. 


2. Overconfidence and Epistemic Rationality 


Epistemic rationality is usually understood to mean hav- 
ing true beliefs about the world (i.e., believing true prop- 
ositions and disbelieving false ones).” In this section, we 
demonstrate how miscalibrated judgments of confidence 
play a role in acquiring and maintaining false beliefs 
about the world and the manner in which one evaluates 
one’s knowledge of the world. 

The feeling of knowing is one type of metacognitive 
judgment we make in evaluating our own knowledge. For 
example, one may be quite sure that one will ultimately 
be able to remember the capital city of Uruguay, even if it 
currently eludes recall. These feelings are often inferences 
that rely on cues such as the amount of information that 
comes to mind (Koriat, 1993), for example, it starts with 
M, is on the coast, and so on. Dunning (2011) argued that 
these cues underlie a set of phenomena he termed “meta- 
ignorance”: an ignorance about the things one does 
not know about. Examples of meta-ignorance include 
overclaiming, in which people claim to have knowledge 
about topics that actually do not exist. This tendency is 
especially pronounced in areas in which people claim to 
be highly knowledgeable (Atir, Rosenzweig, & Dunning, 
2015). For example, participants who rated themselves 
to be knowledgeable about personal finance were more 
likely to claim knowledge about fictitious financial con- 
structs, such as “fixed-rate deduction.” Another example 
is the illusion of explanatory depth, in which people believe 
that they understand how common objects, such as toi- 
lets and toasters, work, when in fact they do not (Fern- 
bach, Rogers, Fox, & Sloman, 2013; Rozenblit & Keil, 
2002). It appears that our familiarity and experience with 
these objects creates a misplaced sense of understanding. 
Dunning (2011) describes numerous instances of situa- 
tions, ranging from the trivial to the calamitous, in which 


Reasoning, Rationality, and Metacognition 


people demonstrate that they are unaware of how little 
they know or how poorly they understand a topic or situ- 
ation, leading them to sometimes dangerous actions (e.g., 
rewiring the electrical system in one’s house). In such 
cases, people have a metacognitive experience feeling 
that leads them to believe that they know more than they 
actually do. 

A third example is the Dunning—Kruger effect, in which 
people demonstrate more confidence in their responses 
than their performance warrants (Kruger & Dunning, 
1999; see also Fischhoff, Slovic, & Lichtenstein, 1977). 
Of particular relevance to the rationality debate is the 
demonstration that this effect extends to tasks designed 
to measure reasoning biases (Pennycook, Ross, Koehler, 
& Fugelsang, 2017). In one study, participants com- 
pleted the cognitive reflection test (Frederick, 2005), which 
consists of three items such as the following: 


A ball and a bat together cost $1.10. The bat costs $1.00 
more than the ball. How much does the ball cost? 


About two-thirds of respondents mistakenly answer 
“10¢,” when a bit of basic algebra would indicate that 
the correct answer is “S¢.” The answer “10¢” comes 
easily to mind, possibly because people miss or misin- 
terpret the “more than” (Mata, Ferreira, Voss, & Kollei, 
2017). As such, the results of this test are commonly 
interpreted as a measure of people’s tendency to respond 
intuitively rather than reflectively (Toplak, West, & Sta- 
novich, 2011). Pennycook et al. (2017) observed a typical 
Dunning-Kruger effect: those who performed well tended 
to underestimate their performance, whereas those who 
performed most poorly believed they did three times 
better than they actually did (for related findings, see 
Mata, Ferreira, & Sherman, 2013). The poor performers 
are therefore “doubly cursed” (Kruger & Dunning, 1999): 
not only do they perform poorly, but they are unaware of 
it and therefore are not in a position to ameliorate their 
performance. 

There are two other relevant points to make about con- 
fidence and overconfidence. First, evidence has emerged 
showing that confidence has a trait-like property, in that 
people who are confident (or overconfident) about their 
performance in one domain also tend to be so in other 
domains (Jackson & Kleitman, 2014; Stankov, Kleitman, 
& Jackson, 2014). Second, as argued above, confidence 
is the arbiter of action. In this case, confidence has been 
found to predict decision-making style (Jackson & Kleit- 
man, 2014; Jackson, Kleitman, Stankov, & Howie, 2016). 
To test this hypothesis, reasoners were asked whether 
they wanted to take action on each decision, for example, 
by submitting an answer (i.e., to count as part of the total 
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score) or administering a treatment for a fictitious disease. 
It was found that confident reasoners take actions that 
are congruent with the decision they made, regardless 
of whether it was accurate. Consequently, those who are 
overconfident make errors of commission (i.e., they act 
when they should not). Conversely, those who are under- 
confident make errors of omission (i.e., fail to act when 
they are correct). 

Up to this point, we have discussed ways in which 
metacognition is a source of epistemic irrationality, by pro- 
ducing monitoring judgments, based on unreliable cues, 
that lead to overconfidence. A more optimistic view comes 
from work on consistency and on metacognitive coher- 
ence. For the latter, read: regularity and predictability; it 
means the sense of fit between environmental cues and 
our knowledge and expectations (Topolinski, 2015, 2018).’ 

We start with work showing that metacognitive judg- 
ments are sensitive to semantic coherence, that is, they 
are sensitive to the rules that govern associations between 
entries in our mental lexicon’ (Sweklej, Balas, Pochwatko, 
& Godlewska, 2014; for a related view, see Betsch & Gl6ck- 
ner, 2010). Much of the work in support of this claim is 
based on a variation of Mednick and Mednick’s (1967) 
Remote Associates Test. The experimental paradigm 
involves presenting participants with word triads. The 
researchers defined triads to be coherent when the three 
words were associated with a fourth word (e.g., “struck,” 
“beam,” and “light” are associated with “moon”). For 
incoherent triads, there is no common remote associate 
(e.g., “car,” “tiger,” and “cream”). Semantic coherence, in 
this context, is a type of fluency cue—it triggers a feel- 
ing of processing ease (or, if lacking, difficulty). Bolte 
and Goschke (2005) observed that participants are able 
to identify semantic coherence of triads at above-chance 
rates, even when they were required to make such judg- 
ments after a brief presentation (namely, in 1.5 seconds; 
for a review, see Topolinski, 2015; but see Ackerman & 
Beller, 2017). 

Although it is not usual to think of them in such a way, 
the judgments of semantic coherence used in this line 
of research are akin to metacognitive monitoring judg- 
ments done regarding one’s own cognitive performance 
(Thompson, 2014). We know that people can make 
judgments regarding their knowledge under similar time 
constraints (Reder & Ritter, 1992). Moreover, both types 
of judgments make an inference about a mental state, 
based on the heuristic cues that generate feeling of flu- 
ency (Topolinski, 2015). In Ackerman and Thompson’s 
(2015, 2017) framework, both are similar to judgments 
of solvability, which, as described above, are prospective 
judgments about whether the participant would be able 
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to solve the problem at hand (see examples in Ackerman 
& Beller, 2017; Lauterman & Ackerman, 2019). 

While consistency and coherence are different aspects 
of metacognitive judgment, they both have to do with 
regularity and predictability versus conflict. Evidence for 
the role of consistency in metacognitive judgments can 
be found in the so-called conflict detection effect (De Neys, 
2014). Many classic reasoning problems are designed 
to be tricky, in that they pit two conflicting answers 
against each other, as in the bat and ball problem above. 
People’s confidence in their answers to such problems is 
lower than it is to control versions in which there is no 
trick (De Neys, Cromheeke, & Osman, 2011; Thompson 
et al., 2011; Thompson & Johnson, 2014). That is, when 
such problems generate a feeling of disfluent processing, 
by permitting two conflicting answers, it is reflected in 
people’s confidence judgments. 

Indeed, Koriat (2012) argued that consistency is the 
major determinant of confidence and also underlies 
fluency. His self-consistency model (SCM) defines self- 
consistency as a mnemonic cue that captures agreement 
among a variety of considerations, including task perfor- 
mance and knowledge. The SCM applies in cases where 
there are two alternatives from which to choose. To 
decide, people are assumed to gather/retrieve informa- 
tion about those two alternatives. Confidence in the 
decision is determined by the consistency of the infor- 
mation that favors the preferred alternative: choices that 
have many pieces of information favoring them, and few 
drawbacks, engender confidence and will be made rela- 
tively more fluently than those that are less consistent. 


3. Normative, Practical, and Bounded Rationality 
in Light of Metacognitive Research 


So far, the literature we reviewed resonates well with the 
heuristics and biases tradition in the psychology of deci- 
sion making (e.g., Gilovich, Griffin, & Kahneman, 2002; 
and see chapter 2.4 by Fiedler, Prager, & McCaughey, 
this handbook). The core idea of heuristics and biases 
is meliorist (a term we borrow from Stanovich, 1999): 
people draw on heuristic mental shortcuts, which in 
everyday life often work well enough, but in many con- 
texts can be spectacularly off the mark, leading to biased 
reasoning and decision making. This approach high- 
lights the notorious normative-descriptive gap: it holds 
that there is a set of norms that ought to be met (such 
as those of the probability calculus or classical logic), but 
actual behavior falls short of them. 

However, there is more to rationality than meeting 
normative rules (or “rationality,” as Evans & Over, 1996, 
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labeled it). Rationality can also be gauged by practical or 
instrumental standards (“rationality,,” in the Evans and 
Over nomenclature): does the thought or decision or act 
lead to achieving one’s goals? Behavior that is norma- 
tively irrational can often be argued to be pragmatically 
rational and vice versa. The term often mentioned in 
this context is “bounded rationality” (Simon, 1982; and 
see chapter 8.5 by Hertwig & Kozyreva, this handbook), 
an idea that turns out to be surprisingly metacogni- 
tive. Simon highlights the inherent cognitive and bio- 
logical limitations on human thinking: we do not have 
unbounded working memory or unlimited attentional 
resources; we do not live forever. Some normative solu- 
tions to rational puzzles are computationally intractable 
in that they would require more than the lifetime of the 
universe to be computed, never mind a human life span. 
Thus, with human rationality necessarily bounded by 
these limitations, the rational approach is, rather than 
to try and find the slippery, often intractable, optimal 
solutions, to scale down to solutions that are just good 
enough: to satisfice, in Simon’s terminology. 

Seen through the lens of metacognitive research, 
satisficing is a type of stopping rule (see Ackerman & 
Thompson, 2015, 2017): it tells us when we can stop 
searching for a better response, because the one we have 
is good enough. Nonetheless, the concept of satisficing 
on its own is rather underspecified. All it tells us is that 
people abort the search prior to achieving a normatively 
valid solution, but it does not tell us the details of the 
underlying psychological mechanisms, for example, 
that conceptual fluency (ease of processing) might trig- 
ger aborting the search regardless of how close to optimal 
the solution is. The unique insight from metacognitive 
research is that satisficing is not some sort of down- 
graded normative processing, where stopping depends 
on some quantitatively partial fulfilment of normative 
parameters. Rather, stopping processing depends on 
cues that are qualitatively different from normative par- 
ameters of the solution and only happen to (modestly) 
correlate with them (see Ackerman, 2014). 


4. Meta-Reasoning and the Rationality of Persistence 


4.1. Are Humans Cognitive Misers? 

One surprising insight that metacognitive research on 
reasoning and decision making can afford us on human 
rationality regards the so-called cognitive miser hypoth- 
esis. Originally proposed in the context of social cogni- 
tion (Fiske & Taylor, 1984, 1991), the cognitive miser 
hypothesis suggests that people are mental Harpagons, 
stingily doling out cognitive resources like the miser in 
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Moliére’s play reluctantly giving up his gold. For exam- 
ple, people are capable of overcoming social stereotyp- 
ing to treat people as individuals rather than cast types, 
but this demands mental effort. Although people pos- 
sess the requisite mental resources, they are often loath 
to invest them, thus succumbing to stereotyping due 
to mental laziness. The cognitive miser hypothesis was 
later adopted into the rationality debate by the melior- 
ist tradition (Evans & Stanovich, 2013; Stanovich, 2009), 
where it transmuted into the more specific idea that 
biases (and therefore irrationality) result from defaulting 
into less effortful processing. 

How sound is the cognitive miser hypothesis? There 
are several presuppositions here: (1) people tend to stop 
searching for solutions prematurely, even though (2) 
they have the necessary cognitive resources to complete 
the search, and (3) this leads to bias and normative irra- 
tionality. It follows that (4) investing more cognitive 
effort should correlate positively with being normatively 
rational. The problem is that each one of these theses is 
suspect. Theses (2) and (3) have both been challenged 
before (for thesis (2), see, e.g., Elqayam, 2012; for thesis 
(3), see Gigerenzer, Todd, & ABC Research Group, 1999). 
Where meta-reasoning research can provide novel insight 
is mainly with (1) and (4). Contrary to widespread con- 
sensus, people often fail to satisfice, and any correlation 
between cognitive investment and normative responding 
is weak at best (for a review, see Ackerman, 2014). 


4.2. Thinking in Vain 

Bounded rationality and satisficing have long been a 
source of contention within the great rationality debate. 
Within the heuristics and biases (aka meliorist) tradi- 
tion, the term “bounded rationality” is used almost as 
a synonym for “error.” In contrast, the Panglossian tra- 
dition (again borrowing a term from Stanovich, 1999), 
particularly in the literature dealing with the fast-and- 
frugal heuristics approach (Gigerenzer et al., 1999), sees 
bounded rationality as the epitome of human rational- 
ity (see also chapter 8.5 by Hertwig & Kozyreva, this 
handbook). In this chapter, we take no side in that par- 
ticular debate, except to note that both sides share the 
presupposition that people do indeed satisfice—which is 
exactly what metacognitive research calls into question. 
There are numerous cases, we learn from metacognitive 
research, in which the opposite is true: people systemati- 
cally waste cognitive effort. 

Nelson and Leonesio (1988) called this labor in vain. 
Metacognitive research shows that when people can self- 
pace their own learning or problem solving, they invest 
the longest time in the most difficult items, those for 
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which they have little chance of success (Ackerman, 
2014; Koriat, Ma’ayan, & Nussinson, 2006; Undorf & 
Ackerman, 2017). They do so even when they them- 
selves judge their chances of success with these items 
to be slim and even when they are offered incentives 
for accuracy (in fact, incentives for accuracy exacerbate, 
rather than mitigate, labor in vain). These participants 
over- rather than underinvest cognitive effort, making 
them cognitive wasters rather than cognitive misers. The 
labor-in-vain hypothesis is diametrically opposed to the 
cognitive miser hypothesis. Importantly, the weight of 
the metacognitive evidence favors labor in vain. 

A major source of evidence comes from Ackerman’s 
(2014) diminishing criterion model (DCM). It shows that 
although reasoners do adjust their levels of aspiration— 
and, as a consequence, their rate of satisficing—to changing 
circumstances, the updating lags far behind diminishing 
resources. In Ackerman’s (2014) experiment 3, for example, 
participants solved the easiest items in about 10 seconds 
with over 90% confidence, whereas the most difficult 
items took about 90 seconds, and even then, participants’ 
confidence was not much higher than 20%. Thus, it took 
a long time for the penny to drop and for participants to 
stop trying for a better (i-e., more confidently held) solu- 
tion. An important component in the DCM is the time 
limit—the maximum amount of time the respondent is 
willing to invest in each task item. Recent studies have 
shown across a large variety of tasks, including memo- 
rizing, various problem-solving tasks, and real-life web 
searches, that people limit the amount of time that they 
spend considering their response, with the aim of not 
wasting time on items they expect to make little progress 
on. However, it is nonetheless the case that the items on 
which the most time is spent have the lowest rates of suc- 
cess (Ackerman, Yom-Tov, & Torgovitsky, 2020; Undorf 
& Ackerman, 2017). Thus, it is not always the case that 
reasoners resort to satisficing, and even when they do, 
they sometimes fail to satisfice sufficiently, even by their 
own lights, when they judge their chances of success to 
be slim. 

The labor-in-vain effect has also been observed in 
people’s information-gathering strategies. This research 
takes place in the fast-and-frugal heuristics research tradi- 
tion, where it is posited that people solve complex prob- 
lems using simple heuristics (Gigerenzer et al., 1999). 
Nonetheless, it has been observed that to make a deci- 
sion, people often gather more information than speci- 
fied by the heuristic. Acommon task asks people to decide 
between a pair of alternatives, such as which stock to 
invest in. They are allowed to gather information about 
each of the options prior to reaching a decision. People 
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often continue to gather information even when they 
already have enough information to discriminate between 
options (see Bréder & Newell, 2008; for reviews, see Hilbig, 
2010). Indeed, people will pay for that information, even 
if it is not helpful (Newell, Weston, & Shanks, 2003). This 
suggests that having additional information, regardless 
of quality, may help people reach a confidence threshold 
where they are comfortable with their decision. 


4.3 Cognitive Investment and Normative Responding 
We now turn to the idea that there should be a positive 
relation between investment of cognitive effort and nor- 
matively correct responding. The main evidence against 
this idea comes from Thompson’s two-response para- 
digm (Thompson et al., 2011). The core method in this 
paradigm consists of presenting participants with (typi- 
cally) classic reasoning, decision-making, or moral judg- 
ment problems and asking them to provide a first quick, 
intuitive response, followed by a second, more consid- 
ered response. Each of the two responses is also accompa- 
nied by rating on a metacognitive confidence scale. For 
the first response, the judgment is the feeling of right- 
ness (FOR) mentioned above. For the second response, 
the judgment reflects a final judgment of confidence. 
The typical finding is that people seldom change their 
response from the first to the second response, but they 
are more likely to do so when metacognitive cues trigger 
low FOR—typically as a result of processing disfluency. 
As noted above, these cues only marginally coincide 
with the normatively correct response. 

The cognitive miser hypothesis would suggest that in 
the rare cases that people change their mind between the 
first and the second response, it should be more often 
in the direction of a normatively correct response than 
the opposite. Actual findings suggest that the connec- 
tion is much more complex. For example, people change 
from a normative to a nonnormative solution almost as 
often as the other way around. While there often is an 
increase in normatively correct responses over time, it is 
typically small (e.g., Bago & De Neys, 2019; Shynkaruk & 
Thompson, 2006; Thompson et al., 2011; Thompson & 
Johnson, 2014). Thus, investing more time and thought 
does not necessarily lead to more normative solutions. 

Similarly, Stupple, Pitchford, Ball, Hunt, and Steel 
(2017) found only a small positive correlation between 
latencies (a potential proxy for cognitive effort) and cor- 
rect solutions for the cognitive reflection test (Frederick, 
2005), which aims to measure the ability to provide nor- 
matively correct responses against the lure of intuitively 
compelling but incorrect responses. Even more interest- 
ing are item-level results. For the bat and ball problem 
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(which requires mental arithmetic to solve correctly), the 
cognitive miser hypothesis was supported: latencies pos- 
itively predicted the percentage of correct responses, and 
correct responses took longest. However, on the lily pad 
problem (which requires little mental arithmetic and is 
more in the nature of an insight item), latencies actually 
negatively predicted normatively correct responses. For 
the latter, intuitively compelling erroneous responses 
were fastest, but nonintuitive erroneous responses were 
slowest—slower than correct responses, in fact. This sug- 
gests that reasoners were investing time and effort, just 
to end up with the wrong answers. 

Taken together, the findings from Thompson et al. 
(2011) and from Stupple et al. (2017) seem to hint at 
an inverted-U-shaped relation between normative cor- 
rectness and cognitive resources. Beyond a certain point, 
additional cognitive resources have little role to play and 
might in fact turn out to be detrimental to normative 
responding. If this is correct, we can outline the bound- 
aries of the cognitive miser hypothesis, which seems to 
have limited scope; beyond that, evidence supports the 
labor-in-vain hypothesis. 


4.4 Rationality and Further Deliberation 

Last, we turn to the contribution of meta-reasoning 
research to a little-explored but significant issue in ratio- 
nality: information gathering. Classic Bayesian decision 
theory provides a normative model for selecting between 
alternative options but is silent on how an option set is 
assembled in the first place. When do we stop looking 
for alternatives? Suppose you are looking for a second 
medical opinion. Does it make sense to look for a third 
opinion? A fourth? Bounded rationality tells us that even- 
tually people will abort the search and that they are likely 
to do so prematurely, but this principle is underspecified. 
Douven (2002) outlined a philosophical model of the 
rationality of further deliberation, suggesting that in any 
given search, the rationality of searching further is deter- 
mined by four parameters: 


1. How satisfactory does the agent find the existing 
options? Ceteris paribus, the better the existing options, 
the lower the value of further search. 

2. What is the agent’s estimate, or prediction, of her 
chance of success in finding a better alternative? 
Ceteris paribus, the stronger the prediction of success, 
the higher the value of further search. 

3. What is the cost of further search? Ceteris paribus, the 
higher the cost, the lower the rationality of further 
search. 


4. What is the time limit for the search? 
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These options can all be operationalized in metacog- 
nitive terms (Ackerman, Douven, Elqayam, & Teodor- 
escu, 2020). The first thing to note is that parameter 
(1) is closely related to Thompson’s feeling of rightness, 
the metacognitive confidence measure. The higher the 
FOR, the more satisfactory the agent will find the exist- 
ing options and the less inclined they will be to search 
further. Evidence for this comes from the two-response 
paradigm and from reasoners’ reluctance to change 
responses accompanied by high FOR. Parameter (2), pre- 
diction of success, might be akin to judgment of solv- 
ability (Ackerman & Thompson, 2017). 

An important insight that comes from meta-reasoning 
research is that parameters (3) and (4)—that is, the cost 
of further search and time limit—are not independent: 
in Ackerman’s (2014) diminishing criterion model, time 
itself is a cost: as time goes by, participants are increas- 
ingly willing to settle for less ambitious goals, in which 
they have lower confidence. 


5. Conclusions 


In sum, we have argued that understanding metacognitive 
processes can advance our understanding of epistemic, 
pragmatic, and bounded rationality. Because cues to con- 
fidence may be unrelated to the accuracy or adequacy of 
decisions, we are often overconfident in our reasoning per- 
formance and lay claim to knowledge about fictitious con- 
cepts. We recast Simon’s (1982) notion of satisficing as a 
metacognitive stopping rule, demonstrating that in many 
circumstances, people fail to satisfice, even when they 
have enough information to make an informed choice. 
Thus, rather than being cognitive misers, people tend to 
labor in vain, spending precious cognitive resources on 
difficult or unsolvable problems. 


Notes 


The first two authors contributed equally to the preparation of 
the chapter. 


1. Nelson and Narens (1990) borrowed the “meta” from Hilbert’s 
“metamathematics” and Carnap’s “metalanguage.” Nelson (1996) 
anchors the term in Tarski’s philosophy of truth, where “object 
language” denotes referents that are not linguistic (e.g., chair, 
happiness), while “meta-language” (and “meta-meta-language,” 
and so on) refers to linguistic terms such as “truth.” Thus, “Snow 
is white” is in the object language, whereas “It is true that snow 
is white” is in the meta-language. The “meta” in “metacognition” 
similarly refers to a referential level of processing: the content of 
thought is the object level and the monitoring of that content is 
the meta-level. 
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2. In philosophy, the role of truth-conduciveness in epistemic 
rationality is moot, particularly the relation between epistemic 
and practical rationality. As Stich (1999) argued, the concep- 
tion of truth in philosophy is too fragmented to allow a single 
normative standard. In psychology of reasoning, Evans (2014) 
advocated a view of epistemic rationality as subservient to 
pragmatic rationality. For a recent defense of the connection 
between truth-conduciveness and pragmatic rationality, see 
Schurz (2014) and Schurz and Hertwig (2019), although we 
note that Schurz concedes that this view is somewhat under- 
mined by placebo effects (beneficial false beliefs). Here we stick 
to the classic notion of epistemic rationality based on truth- 
conduciveness, as this is the view that dominates most of the 
metacognitive literature, but the reader is invited to keep this 
caveat in mind. 


3. The metacognitive conception of coherence is therefore 
distinct from the Bayesian one in than it does not rely on con- 
formity to the probability calculus. 


4. The term “semantic coherence” is borrowed from linguis- 
tics, where it means the extent to which words adhere to a 
word formation rule, so that they are predictable given the rule 
(Aronoff, 1976, p. 38). 
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Summary 


Classical game theory extends rational choice theory to 
interactions among several actors. This chapter introduces 
the formal language of classical noncooperative game 
theory in a self-contained way. Emphasis on the exten- 
sive form of games and its presuppositions clarifies game 
theory’s scope and limits as a modern /ingua franca of the 
behavioral sciences. The discussion of rational behavior in 
games centers on the equilibrium concept and its most 
important refinement, subgame perfection. A final section 
relates this discussion to the interpretation of classical 
game theory in terms of “reasoning about knowledge.” 


1. Outline of the Chapter 


“Classical game theory” extends rational choice theory to 
interactions among several actors. We use the term to dis- 
tinguish our topic from related but distinct fields like epis- 
temic, evolutionary, and behavioral game theory. We focus 
on noncooperative game theory, which is more funda- 
mental than cooperative game theory (Harsanyi & Selten, 
1988, chapter 1). Our presentation is mostly ahistorical; 
for historical accounts, see Weintraub (1992) and Leonard 
(2010). References were primarily selected to provide con- 
venient entry points to the literature. For in-depth cover- 
age of mathematical aspects, we refer readers to Fudenberg 
and Tirole (1991) and Maschler, Solan, and Zamir (2013). 

Section 2 approaches games as mathematical struc- 
tures. Section 3 focuses on applying game theory with 
the aim of explaining or predicting interactions among 
several actors. Section 4 explains, and comments on, the 
tension within applied game theory between the inter- 
pretation we present and an alternative interpretation in 
terms of “reasoning about knowledge.” 


2. Game-Theoretic Language 
2.1. The Rules of a Game 


Primitive terms of mathematical game theory are “player,” 
“move,” “payoff,” and “probability.” The use of these 


terms is restricted by certain formal requirements but 
otherwise unspecified. Depending on the application, a 
player can be an individual person, a group of persons 
(say, a firm), or even a bacterium. There are applications 
where a single person is modeled as a set of different play- 
ers or where a player stands for logical operations. Prob- 
abilities satisfy the Kolmogorov axioms (see chapter 4.1 by 
Hajek & Staffel, this handbook), but their interpretation is 
left open. Payoffs are real numbers, but it remains unspeci- 
fied what these numbers represent. For ease of exposition, 
we nevertheless speak of players “making,” “selecting,” or 
“choosing” moves and “receiving” or “earning” payoffs; 
moreover, we cast a female as our generic player and as 
players 1 and 3, as well as a male as player 2. 

The definition of a game as a mathematical struc- 
ture, often called “the rules of the game,” specifies all 
moves that are possible in the game, implying that rule- 
breaking moves cannot exist. A “play of the game” is a 
sequence of moves, each made by a specific player. Each 
possible play of a game comes with a “payoff profile,” 
that is, an assignment of one payoff to each player. 

The rules of a game are summarized in the game’s 
“extensive form,” also called “game tree.” Figure 9.1.1a, 
for instance, describes a game that begins with player 
1 choosing between moves A and B. If she chooses A, 
player 2 must choose between W and X; if player 1 
chooses B, player 2 must choose between moves Y and Z. 
After moves X, Y, or Z of player 2, the game ends. After 
move W, player 1 must choose between moves C and 
D, after which the game ends. A “path” is a succession 
of nodes and branches. Each complete path through 
the tree, for instance, A-W-D or B-Y (plus the relevant 
nodes), represents a possible play of the game. The asso- 
ciated payoff profile is displayed at the end of the path; 
thus, A-W-D is associated with the payoff profile (4, 6), 
that is, player 1 earns 4 and player 2 earns 6. 

All game trees we consider begin with a unique ini- 
tial node (indicated by a circle) and end with a finite 
number of final nodes represented by payoff profiles. 
Bullets represent intermediate nodes. Branches connect 
each node—except the initial node—with its immediate 
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Figure 9.1.1 


(a) A strictly sequential game. The pure strategies of player 1 are AC, BC, AD, and BD. Those of player 2 are WY, XY, WZ, and XZ. 
(b) The pure-strategy profile (BC, XZ), illustrated by the highlighted moves, implies the path B-Z with associated payoff profile (3, 4). 


predecessor. Each node of a game tree is reached from 
the initial node through a unique path. The games we 
consider are finite in the sense that there are finitely 
many complete paths, each of finite length. 

Each nonfinal node is assigned to one specific player. 
Names of players are numbers appearing (underscored) 
close to their nodes. There may, however, exist passive 
players without nodes who receive payoffs but cannot 
move. The branches emerging from a player’s node cor- 
respond to different moves available to the player at the 
node. Names of players’ possible moves are capital let- 
ters appearing at the respective branches. 

The path reaching a node corresponds to the history 
of play up to the node; therefore, this path is also called 
the node’s “history.” A complete path ends with the pay- 
off profile associated with the corresponding play of the 
game. Payoff profiles list payoffs in the order of the play- 
ers’ names, from 1 to n. 

A situation in a game where a player must move is 
called “information set.” In figure 9.1.1a, all informa- 
tion sets are singleton sets containing exactly one node; 
in this case, distinguishing between nodes and informa- 
tion sets is inessential. Each singleton information set, 
or node, of a player comes with its own set of moves. 
Thus, player 2 chooses among W and X at his left node 
and between Y and Z at his right node; his total number 
of different moves is 4. 

This is different in figure 9.1.2a, where the two nodes 
of player 2 belong to the same information set (as indi- 
cated by the broken-line connection). The two moves 


possible at this two-element information set, W and X, 
are displayed at each of the two nodes; however, the two 
branches labeled W stand for a single move, as do the two 
branches labeled X. In total, player 2 has only two differ- 
ent moves (in contrast to figure 9.1.1a, where he has four). 
In applications, information sets are used to describe situ- 
ations where a player is unable to observe (or remember, 
or notice in some other way) what had happened before 
she was called upon to move. The game, then, begins with 
player 1 choosing between A and B, followed by player 2 
choosing between W and X. In case of A and W, player 1 
chooses for a second time, between C and D. 

Generally, each node of a player belongs to exactly 
one information set; any number of nodes of a player 
can belong to the same information set. At each of 
her information sets, a player has at least two possible 
moves, of which exactly one must be chosen if the play 
of the game reaches (a node of) the information set. 
The possible moves at an information set are displayed 
at every node of that information set, with their names 
indicating which moves at the different nodes are the 
same moves (so that exchanging names at one node but 
not at the others leads to a different game). 

Ina “simultaneous” game, each player has at most one 
information set, and each complete path runs through 
all information sets (as in figure 9.1.3). A nonsimultane- 
ous game is often called “sequential,” although it need 
not be “strictly sequential” in the sense that each infor- 
mation set is a singleton set (as in figure 9.1.1). Most 
games are neither simultaneous nor strictly sequential 
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(a) A game that is neither strictly sequential nor simultaneous. Player 2 has only two pure strategies, W and_X, since both his nodes 


belong to the same information set, meaning that he must choose the same move, W or X, at both nodes. (b) The pure-strategy 


profile (AD, X), illustrated by the highlighted moves, implies the path A—X with associated payoff profile (2, 1). 


(see, e.g., figures 9.1.2, 9.1.4, and 9.1.5). Strictly sequen- 
tial games are often (rather confusingly) called “games of 
perfect information.” 


2.2 Strategies and Equilibrium 

A pure strategy of a player is a function assigning to 
each of her information sets exactly one of the possible 
moves. In figure 9.1.1, each player has two information 
sets with two possible moves each; therefore, each player 
has four (2 x 2) pure strategies. We denote pure strategies 
by a sequence of letters spelling out the moves selected 
at each information set (in the order from left to right 
and top to bottom). The four pure strategies of player 
1, then, are AC, AD, BC, and BD, while player 2’s four 
strategies are WY, WZ, XY, and XZ. 

A pure-strategy profile is a tuple of pure strategies, 
one for each active player, again in the order of players’ 
names. In figure 9.1.1, a pure-strategy profile determines 
a unique complete path through the game tree, that is, a 
unique play of the game and, therefore, a unique payoff 
profile. For instance, the profile (BC, XZ) implies the path 
B-X with payoff profile (3, 4) (see figure 9.1.1b). Not each 
move selected by the strategies occurs in the play of the 
game generated by (BC, XZ): player 2 is not called upon 
to move at his left-hand node; nevertheless, his strategy 
selects X. Moreover, player 1’s own choice of B prevents 
that she is called upon to move at her second node, where 
her strategy selects C. Yet, distinguishing between strate- 
gies like BD and BC turns out to be relevant. 


In figure 9.1.2, player 2 has only two pure strategies, 
called, like the moves, “W” and “X” (accepting a minor 
ambiguity in the notation). Figure 9.1.2b illustrates the 
strategy profile (AD, X). In contrast to figure 9.1.1, player 
2 has no pure strategy that discriminates, through its 
assignment of moves, between the different histories 
preceding his two nodes. Nonsingleton information sets, 
then, decrease the number of pure strategies. 

The “strategic (normal) form” of a game is a list of all 
pure-strategy profiles together with the resulting payoff 
profiles. For two-player games, the strategic form is writ- 
ten in the form of a cross-tabulation. Table 9.1.1 displays 
the strategic forms of the games of figures 9.1.1 and 9.1.2. 
Strategies differing only at nodes that are out of reach 
due to the strategies themselves (like BC and BD in fig- 
ure 9.1.1) generate identical entries in the strategic form. 
Different games can have the same strategic form. 

Strategies can be stochastic. Let {Fi,..., E,} be a 
finite set. A “mixture” of the set’s elements, written as 
pick; ® p2°E, ®+++@® pueEn, is a probability distribution 
assigning to each E, a probability p,x=O with Sx px = 1. 
Probability-O elements can be dropped; the degenerate 
cases 1° Ex are identified with E;. Mixtures of elements 
from different sets are always stochastically independent. 

A “mixed strategy” of a player is a mixture of her pure 
strategies. A “behavioral strategy” of a player is a func- 
tion assigning to each of her information sets a mixture 
of the moves at that information set, called a “mixed 
move.” Mixed and behavioral strategies include the pure 
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Table 9.1.1 
The strategic forms of the games of figure 9.1.1 (left-hand side) 
and figure 9.1.2 (right-hand side) 


2 2 
WY WZ XY XZ w x 
AC | 501 5,0 | 2. 2,4 ACS) 6. 4 
AD |4,6|4,6]2,1]2,1 AD |4,6]2,1 
BC |0,3|3,4|0,3 | 3,4 ‘ BC |0,3|3,4 
BD |0,3|3,4/0,3|3,4 BD | 0,3 | 3,4 


Note: Underscored payoffs indicate best replies to the other 
player’s pure strategies. A strategy profile leading to a cell 
where all payoffs are underscored—for example, (AC, XY) on 
the left—is an equilibrium. 


strategies as degenerate cases. Further extensions of the 
strategy set, such as correlated (i.e., stochastically depen- 
dent) strategies (Maschler et al., 2013, chapter 8), go 
beyond classical game theory. 

The payoff profile associated with a profile of mixed or 
behavioral strategies is, in general, a profile of expected 
(values of) payoffs (see also section 2.4 below). Subse- 
quently, the term “payoff” covers “expected payoff”; we 
add “expected” only for special emphasis. 

Exogenous stochastic events are represented by a 
player without payoffs (usually called “Nature”), play- 
ing, at each of its nodes, a specific mixed move. Although 
this extension has many important applications (see, 
e.g., Fudenberg & Tirole, 1991, parts III & IV, on so-called 
Bayesian games), we can disregard it for present purposes. 

A “solution concept” selects, for each game in some 
class of games and given strategy sets (pure, mixed, or 
behavioral), a set of strategy profiles as solutions. The basic 
solution concept of classical game theory is the Nash equi- 
librium (subsequently, just “equilibrium”). 

An “equilibrium” is a strategy profile where the strat- 
egy of each player is a best reply to the “complementary 
strategy profile,” that is, the tuple of strategies of the other 
players (Nash, 1950). “Best reply” of a player means: no 
other strategy of the player achieves a higher payoff—or 
expected payoff, if relevant—against the complementary 
strategy profile. In an equilibrium, then, each player's 
strategy maximizes the player’s payoff given the other 
players’ strategies; no player’s payoff can be increased by 
changing only the player’s own strategy while leaving the 
complementary strategy profile unchanged. 

We focus on equilibria of two-player games since it 
usually does not matter whether the complementary 
strategy profile appearing in the definition of equilib- 
rium involves one or several players. 
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2.3 Pure-Strategy Equilibrium 

Figure 9.1.3 shows the general form and several exam- 
ples of simultaneous games with two players where each 
player has two moves (simultaneous 2 x 2 games). We find 
the pure-strategy equilibria of these and all other finite 
games by underscoring, in the strategic form, those pay- 
offs of each player that are associated with best replies to 
the other player’s pure strategies. A pure-strategy equilib- 
rium is associated with a cell where all payoffs are under- 
scored (see table 9.1.1 and figures 9.1.3 and 9.1.4). 

In most games, a player’s equilibrium strategy can- 
not be determined independently of the equilibrium 
strategies of the other players. The Prisoner’s Dilemma 
(figure 9.1.3a) is the most prominent exception: player 
l’s strategy U is strictly dominated by her strategy D, 
that is, D yields a higher payoff than U, no matter which 
(pure or mixed) strategy is played by player 2. Similarly, 
player 2’s strategy L is strictly dominated by his strategy 
R. Therefore, the only equilibrium is (D, R). 

An equilibrium is “strict” if and only if (iff) each 
player has exactly one best reply to the complemen- 
tary strategy profile. With the exception of (U, L) in fig- 
ure 9.1.3d, the equilibria shown in figure 9.1.3 are strict. 

A pure-strategy equilibrium in a game without moves of 
Nature selects a unique play of the game, that is, a unique 
complete path through the game tree, called an “equilib- 
rium path.” In an equilibrium with mixing (by Nature or 
by conventional players), the term “equilibrium path” 
denotes the set of complete paths with positive probability. 


2.4 Mixed-Strategy Equilibrium 

As shown by Matching Pennies (MP; figure 9.1.3b), there 
are games without pure-strategy equilibria. Yet, all finite 
games have at least one mixed-strategy equilibrium (Nash, 
1950). 

A profile of mixed strategies yields a probability distri- 
bution on the set of all payoff profiles and, thus, a profile 
of expected payoffs. For the general game of figure 9.1.3, 
we find that the mixed-strategy profile (p°U © 1-peD, 
qoL ®1-qeR) yields payoffs pqx: + p(1-q)x2 + (1-p) 
qx3+ (1-p)(1-q)x4 for player 1 and pqyi + pl-4q)y2 
+ (1-p)qy3+ (1-p)(1 - q)ys for player 2. 

The definition of equilibrium remains unchanged. 
In MP, for instance, we find the unique mixed-strategy 
equilibrium (42°U © %eD, %eL © %oR) with payoff pro- 
file (0, 0). 

Mixing maximizes a player’s payoff iff all pure strat- 
egies played with positive probability are best replies 
to the complementary strategy and, therefore, yield 
the same payoff. This insight has three important con- 
sequences, which we spell out for simultaneous 2x2 
games but which hold, mutatis mutandis, for other games 
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4,0 


(a) Prisoner’s Dilemma: X3>X1>X4>X2, 


Yr Vi > ya > ys. 


(C) Battle of the Sexes: x, >x4>2%3=22, ya >i >y3= Jr. 


Figure 9.1.3 
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Xo5 V2 


(b) Matching Pennies (or Inspection Game): x, = 


Ny > Xy = NX3y V2 = Vs > Vi = Vee 


(d) Nameless game. 


Below the general form, extensive and strategic, of a 2x 2 simultaneous game, three well-known named games (general structure 


indicated) and a nameless game are displayed. 


where mixing occurs in equilibrium. First, no equilib- 
rium where mixing occurs is strict. Second, if both play- 
ers mix in equilibrium, one player’s mixed strategy must 
equalize the expected payoffs of the other player’s pure 
strategies (indicating how to compute the equilibrium). 
Third, small changes in one player’s payoffs affect only 
the other player’s equilibrium strategy. For instance, mod- 
ifying MP by adding 1 to player 1’s payoff in (U,L) leads 
to the new equilibrium (2° U © %°D, %°L © %oR). 

Pure-strategy equilibria are special mixed-strategy equi- 
libria. Accordingly, “Battle of the Sexes” (figure 9.1.3c) 
has three mixed-strategy equilibria: (U,L), (D,R), and 
(AcU® %eD, BeL @ BR). 


2.5 Subgame Perfection 
Most games have several equilibria. So-called refine- 
ments of the equilibrium concept select subsets of these 


equilibria. We consider only the most important refine- 
ment, subgame perfection (Selten, 1965, 1975). 

Subgame perfection is concerned with moves at infor- 
mation sets off the equilibrium path (i.e., information sets 
reached with probability O in equilibrium). Since these 
moves do not affect the players’ equilibrium payoffs, they 
can fail to maximize payoffs from the perspective of the 
information set where they are taken. Examples are player 1’s 
move D and player 2’s move X in the equilibrium (BD, XZ) 
in figure 9.1.1 (not highlighted; see also table 9.1.1). Sub- 
game perfection aims at eliminating equilibria containing 
such non-payoff-maximizing moves. 

Payoff maximization from the perspective of an infor- 
mation set requires that subsequent moves and, in the 
case of non-singleton sets, preceding moves are taken 
into account. Consider the equilibrium (XU,R) in fig- 
ure 9.1.4, All moves except the initial move X of player 1 
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Figure 9.1.4 
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2 
pe R 
SU 4,2 0,0 
SD 0,0 2,4 
1 
XU 3,3 3,3! 
XD 3,3 3,3 


Player 1 decides whether to start (S) a subgame (type “Battle of the Sexes”) or to exit (E) instead. The strategic form shows three 


pure-strategy equilibria; only (XU,R) (highlighted) is not subgame-perfect. An additional subgame-perfect equilibrium is (XM, N) 


with M = %°U©@ %eD and N = %eL © BR. 


are off the equilibrium path. Given player 1’s subsequent 
move U, player 2’s move R does not maximize his payoff. 
Given player 2’s preceding move R, player 1’s move U 
does not maximize her payoff. 

Subgame perfection addresses this interdependency 
problem by considering the section of the game begin- 
ning at player 2’s node as a separate game, called a “sub- 
game,” requiring that the equilibrium of the complete 
game be consistent with some equilibrium of the sub- 
game. The subgame has three equilibria, (U,L), (D,R), 
and (*4°U@ %eD, %4eL © %eR), with payoff profiles 
(4,2), (2,4), and (%,%), respectively. In all three, both 
players’ information sets are on the equilibrium path, 
implying payoff maximization by both. 

In the complete game of figure 9.1.4, strategy S maxi- 
mizes player 1’s payoff iff her payoff in the subgame is 
at least 3. Accordingly, there are three subgame-perfect 
equilibria of the complete game: (XD, R), (SU,L), anda 
behavioral-strategy equilibrium (XN, M) with N=%°:U@® 
%oD and M=%cL © ¥eR. The equilibrium (XU,R), in 
contrast, is not subgame perfect since (U,R) is not an 
equilibrium of the subgame. 

Generally, a section of a game tree is a subgame iff it 
starts with a singleton information set, contains all suc- 
cessors to its starting node down to the final nodes with 
the payoffs, and contains no incomplete information 
sets (i.e., any non-singleton information set is either 
completely included or completely excluded). The com- 
plete game is an (improper) subgame of itself. A subgame 


considered in isolation, then, satisfies the definition of 
a game. 

A strategy profile is a subgame-perfect equilibrium of 
a game iff, for all subgames, the profile’s restriction to 
the subgame is an equilibrium of the subgame. Subgame- 
perfect equilibria are, in general, equilibria in behav- 
ioral strategies because mixed strategies cannot describe 
mixed moves at information sets that are, according to 
the strategy itself, reached with probability 0 (Fudenberg 
& Tirole, 1991, pp. 87-88, esp. figure 3.10). Every finite 
game has at least one subgame-perfect equilibrium in 
behavioral strategies (Selten, 1965). 

In simultaneous and other games without proper sub- 
games (figures 9.1.3 and 9.1.5), each equilibrium is subgame 
perfect by definition. In all other games, subgame-perfect 
equilibria are found by (a generalized version of) “backward 
induction.” This solution method begins with finding one 
equilibrium for each of the “smallest” subgames (those that 
have no proper subgames themselves). It then goes on to 
more and more encompassing subgames, always preserv- 
ing the equilibria of already solved subgames when solving 
the more encompassing ones (as in figure 9.1.4’s game 
above). 

In strictly sequential games, each nonfinal node is also 
the first node of a subgame. The smallest subgames begin 
at the last player nodes and have only one active player. In 
figure 9.1.1b, backward induction first selects C by player 
1 and Z by player 2 at the two last nodes, then X at the left 
node of player 2, and, finally, B at the initial node. Thus, 
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the unique backward-induction solution is (BC, XZ). This 
solution is an equilibrium (see table 9.1.1), and its restric- 
tion to any proper subgame is an equilibrium of the 
subgame—for instance, (C,X) for the subgame beginning 
at player 2’s left node. Therefore, the backward-induction 
solution is a subgame-perfect equilibrium. 

In figure 9.1.2, the only proper subgame begins at 
the second node of player 1 and has a single equilibrium 
where player 1 chooses C. Of the two pure-strategy equi- 
libria of this game (see table 9.1.1), then, only (BC,X) is 
subgame perfect. 

Yet, subgame perfection still allows for non-payoff- 
maximizing moves. Figure 9.1.5 illustrates two strat- 
egy profiles of Selten’s (1975) “Horse Game.” Both are 
subgame-perfect equilibria: no player can achieve a higher 
payoff by unilaterally changing her strategy, and there 
are no proper subgames. Yet, in figure 9.1.5a, the move of 
player 2 obviously fails to maximize his payoff from the 
perspective of his information set. 

The problem is that subgame perfection addresses 
the issue of non-payoff-maximizing moves only indi- 
rectly, by considering subgames, which do not exist in 
the Horse Game. This suggests a more direct approach: 
require that each player at each information set maxi- 
mize her payoffs. In the case of singleton information 
sets, this works. However, consider the equilibrium in fig- 
ure 9.1.5b, where player 3’s information set is not reached. 
Since this is a non-singleton information set, identification 
of a payoff-maximizing move would require an appeal to 
preceding moves reaching the information set—but there 
are none. Mathematically, payoff maximization at player 
3’s information set requires a probability distribution on 
the set of nodes, but any such distribution is arbitrary. 
In more complicated games, a similar problem occurs if 
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= Re 2 R 1 
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0 3 0 4 
0 2 0 4 
0 2 1 0 


Figure 9.1.5 
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several preceding moves reach a non-singleton informa- 
tion set off the equilibrium path. These problems provide 
the starting point for competing refinements beyond sub- 
game perfection (e.g., Maschler et al., 2013, chapter 7). 


3. Applying Game Theory 


In applied game theory, game-theoretical models are 
used to explain and predict behavior. A model is a theory 
together with the description of a situation, real or hypo- 
thetical, to which the theory is applied in order to deduce 
predictions or explananda (Bunge, 1973, pp. 97-99). 

We restrict attention to human behavior. In classi- 
cal game theory, the relevant theory of human behav- 
ior is rational choice theory (RCT; cf. Sugden, 1991, and 
chapter 10.4 by Raub, this handbook), which is applied 
to situations described by a game tree. Players are indi- 
vidual persons. Payoffs are utilities representing players’ 
preferences. Probabilities are either physical probabili- 
ties known to all players or players’ subjective degrees of 
belief. In the latter case, the probabilities are, quite arbi- 
trarily (Sugden, 1991, p. 768), assumed to be the same 
for all players (the “common prior assumption”). 


3.1 Rationality and Preferences 
Applied game theory assumes that players are rational 
actors in the sense of RCT, know the game tree, and have 
true beliefs about the strategies played by the other players. 
According to RCT, an actor is rational in the sense 
that she has, for any set X of mutually exclusive and 
jointly exhaustive options, a complete preference order- 
ing described by a complete and transitive relation > on 
X, called “weak preference” (see chapter 8.1 by Griine- 
Yanoff, this handbook). For a, bEX, a=b means either 


1 
= i 2 R 1 
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L R L R 
0 3 0 4 
0 2 0 4 
0 2 1 0 


(a) Selten’s Horse Game with equilibrium (D,R, R) highlighted. Player 2 does not maximize his payoffs from the perspective of his 


information set. (b) Selten’s Horse Game with equilibrium (R, R, L) highlighted. It is unclear what it would mean for player 3 to 


maximize her payoffs from the perspective of her information set. 
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that she strictly prefers a to b (symbolically, a> b) or that 
she is indifferent between the two (symbolically, a~b). A 
real-valued function u defined on X is said to represent 
the preference relation (and is called “utility function”) iff, 
for all a, b EX, u(a) = u(b) implies a= b and vice versa. This 
ensures that u(a) > u(b) iff a> b, and u(a) = u(b) iff a~b. 

Under the standard (“revealed-preference”) interpre- 
tation, preferences only summarize behavioral disposi- 
tions: a> b means that the actor would not choose b if 
she had to choose from a subset B of X with a, b€B. 
This subjunctive conditional exhausts the meaning of 
the strict preference relation. It implies that the actor 
would always maximize her utility: she would choose 
c&B only if u(c) = maxyegu(x). If B contains several 
utility-maximizing options, the theory implies that the 
actor would choose one of them; apart from that, her 
behavior would be indeterminate. 

This interpretation implies that RCT considers nei- 
ther decision making nor deliberation or reasoning as 
mental processes; it is only concerned with the resulting 
choices. Neither utilities nor preferences are reasons for, 
or causes of, choices; they just summarize hypothetical 
choices, that is, behavioral dispositions. 

Preferences themselves may result from all kinds of 
motivations—materialistic or nonmaterialistic, selfish 
or unselfish, and so on—but they are “all-inclusive”: 
all factors influencing choices are already taken into 
account (Binmore, 1994, pp. 104-109). Although sub- 
jective well-being may motivate an actor’s choices, her 
preferences may reflect other factors as well (e.g., moral 
considerations). Therefore, RCT is not wedded to the 
assumption that an actor strictly preferring a to b is sub- 
jectively better off if she gets a than if she gets b (Haus- 
man, 2012, pp. 81-83). 

In addition to some theory of motivation, predictions 
or explanations also require hypotheses (lacking in RCT) 
that specify the conditions under which players are ratio- 
nal and have true beliefs about the situation and other 
players’ strategies—for instance, shared cultural back- 
ground (Meyerson, 2009) or experience and sufficiently 
high stakes (Binmore, 2007, chapter 1). Given these pos- 
sible specifications, RCT is not a single theory but a family 
of distinct theories relying on a common mathematical 
language. Applied game theory inherits this feature. 


3.2 Preferences in Games 

In game theory, actors are players who choose moves 
at information sets. This has two implications. First, 
the objects of choice are moves at information sets, not 
strategies. If a player has several information sets, she 
cannot choose a strategy in a single act of choice. Each 
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information set, whether reached during play or not, 
requires its own (possibly hypothetical) choice act. 

Second, a player’s utilities attach to complete paths, 
which typically contain moves by other players. There- 
fore, the objects of preferences, namely, complete paths, 
differ from the objects of choice, namely, moves at 
information sets. As such, this is not problematic: given 
beliefs about other players’ strategies, choosing moves 
at information sets amounts to choosing among com- 
plete paths (see section 3.5 below). However, it becomes 
a problem if the question of who chooses matters to the 
players: in these cases, preferences over complete paths 
defined in terms of hypothetical choices may not exist 
(Hausman, 2012, pp. 31-33). 

Consider, for instance, a game where Eve must accept 
(A) or decline (D) Adam’s marriage proposal. Assume 
that Adam wishes her to accept; in this sense, he prefers 
the A-path to the D-path. However, this is not the same 
as saying that, hypothetically, Adam would choose the 
A-path over the D-path if it were his choice. If, plausibly, 
the distinction between “the X-path chosen by Eve” and 
“the X-path chosen by Adam” matters to Adam, he can- 
not have preferences in the revealed-preference sense over 
paths chosen by Eve. 

If, in contrast to this example, the objects of a play- 
er’s desires are, physically or in the player’s mind, sepa- 
rable from the paths (e.g., monetary gains), it does not 
matter who chooses. Preferences over complete paths 
derive from preferences defined in terms of hypotheti- 
cal choices over these separable objects. This separabil- 
ity assumption is sometimes called “consequentialism” 
(Hausman, 2012, pp. 42-45). 


3.3 Uncertainty and Beliefs 

Extending RCT to “choice under uncertainty” (see also 
chapter 8.2 by Peterson, this handbook) requires a con- 
sequentialist distinction between actions, states, and 
consequences that are separable from actions and states 
(see table 9.1.2). The elements of each set (of actions, 
of states, and of consequences) are mutually exclusive 
and jointly exhaustive. An actor’s set of actions describes 
the options among which she must choose. The set of 
states describes all aspects of the situation not under her 
control. States differ in those aspects about which she is 
uncertain. If an action is taken in some state, one of the 
consequences obtains. 

Von Neumann and Morgenstern’s (2004) theory of 
choice under risk assumes that a physical probability 
distribution on the set of states is given and known to 
the actor. Each action, then, implies a probability distri- 
bution on the set of consequences, called a “lottery.” If 


Classical Game Theory 


the actor has a complete preference ordering on the set 
of all conceivable lotteries, and if her preferences satisfy 
several further axioms, there exists a real-valued utility 
function on the set of consequences such that the lot- 
teries’ expected utilities represent the actor’s preferences 
over actions. 

Yet, if a player has nonconsequentialist motivations, 
her preferences may violate the axioms of the theory, and 
an expected utility representation may not exist (Bradley 
& Stefansson, 2017). Examples are von Neumann and 
Morgenstern’s (2004, pp. 629-630) “utility of gambling” 
and certain procedural-fairness motivations (Diamond, 
1967; Verbeek, 2001). 

In Savage’s (1954) theory of choice under radical uncer- 
tainty, no physical probabilities are given; the probabili- 
ties are subjective. Both the utility function on the set of 
consequences and the subjective probabilities are derived 
from a preference ordering on the set of all conceivable 
actions (considered as functions assigning consequences 
to states). Again, the (subjective) expected utilities of the 
actions represent the actor’s preferences. 

In a strict revealed-preference interpretation of Sav- 
age’s theory, the expected utilities of actions just summa- 
rize the actor’s behavioral dispositions. Usually, however, 
the subjective probability of a state is interpreted as a per- 
sonal degree of belief that the state obtains. In contrast to 
the utilities of the consequences, beliefs are mental states. 

This interpretation of beliefs as mental states, on the 
one hand, and utilities of complete paths as expression 
of behavioral dispositions, on the other hand, is typi- 
cal of classical game theory, whether or not beliefs are 
explicitly introduced as subjective probabilities. 


3.4 Information, Causality, Time, and Memory 
The game tree describes the information flow in plays 
of the game. Players called upon to move can observe at 


Table 9.1.2 
Decision problem with four actions Aj, three states S;, and five 
consequences C, 


States 


Si S2 S3 


Ai Cy (on Cy 


Az Cy Cz C3 


Actions 
A3 Ce C2 C, 


Ag C4 Cy Cy 


Note: Uncertainty about the states implies uncertainty about 
the consequences of actions. 
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which information set they are, but, except for singleton 
information sets, not at which node. Since the transmis- 
sion of information requires a causal link, the game tree 
describes at least some aspects of the causal relations rel- 
evant in the game. However, Newcomb’s problem (recast 
as a game) shows that some conceivable causal structures 
cannot be described with the help of a game tree (Albert 
& Heiner, 2003; Binmore, 1994, pp. 242-256). 

The flow of information also describes the flow of 
time: if a player observes a move, the move must have 
occurred earlier. Where no observations take place (as, 
e.g., in simultaneous games), the game tree is consistent 
with any timing. 

Applied game theory mostly assumes “perfect recall”: 
players forget neither their own moves nor anything they 
have known or observed before (Fudenberg & Tirole, 
1991, p. 81). But models of imperfect recall are possible. 

In figure 9.1.6a, player 1 forgets her initial move. It 
seems natural to assume that, at her second information 
set, she behaves as if she were an independent third player 
with player 1’s preferences. The resulting equilibria—for 
instance, (L,DX,L) (three-player notation) or (LL,DX) 
(two-player notation)—assume that, as the third player, 
she has correct beliefs about her first-player strategy, 
although she forgot what she did in that role (which is 
not implausible with respect to routine behavior). 

In figure 9.1.6b, player 1 does not know whether she 
has already moved (“absentmindedness”; Piccione & 
Rubinstein, 1997). Her equilibrium beliefs are the sub- 
ject matter of philosophy’s “sleeping beauty problem” 
(Cisewski, Kadane, Schervish, Seidenfeld, & Stern, 2016). 


3.5 Equilibrium and Beliefs about Other Players 

A player’s behavioral dispositions in a game are described 
by a behavioral strategy. A behavioral strategy is a com- 
plete contingent plan of action, that is, a list of sub- 
junctive conditionals stating, for each information set 
of the player, which (possibly mixed) move the player 
would choose at this information set. According to RCT, 
a player would choose a move only if the move maxi- 
mized her expected utility given her beliefs at the rel- 
evant information set. 

The concept of equilibrium among rational players 
originates from economics; traditionally, it involves the 
assumption that everybody’s beliefs are correct. If one 
assumes that all beliefs everywhere in the game are cor- 
rect, RCT implies that all moves, those actually taken in 
the game as well as the hypothetical moves off the equi- 
librium path, maximize the relevant player’s expected 
utility from the perspective of the information set where 
they are supposed to be made. 
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1 


Figure 9.1.6 


Max Albert and Hartmut Kliemt 


(a) At her second information set, player 1 has forgotten her initial move. (b) Player 1 does not know whether she moved before 


(“absentmindedness”). 


The game of Blackmail in figure 9.1.7 illustrates the 
basic argument. Eve (player 1) knows some damaging 
fact about Adam (player 2). She can keep silent (S) or, by 
threatening to reveal the fact, try to extort money from 
him (X). If Eve chooses X, Adam can give in and pay (P) 
or report Eve to the police (R), in which case she goes to 
jail but the damaging fact is revealed, to Adam’s distress. 
In the strategic form, we find two pure-strategy equilib- 
ria, (S,R) and (X,P). 

Players’ utilities indicate how, given the opportunity, 
they would choose among complete paths. If Eve chose 
X, Adam could choose between complete paths X-R 
and X-P; preferring X-P, he would choose P. Since Eve 
has correct beliefs about what Adam would do and pre- 
fers the complete path X-P to the complete path S, she 
chooses X. Therefore, players play in accordance with 
the subgame-perfect equilibrium (X, P). 

More generally, a player’s beliefs are conditional on 
information sets, that is, they take into account what the 
player knows at the information sets. Players anticipate 
their own beliefs and, therefore, their own moves. Even 
if a player believes that she would never reach a certain 
information set, her beliefs about how other players 
would move after this information set had been reached 
are the beliefs she would have if this—in the player’s 
eyes, impossible—event happened. The requirement 
that beliefs are conditional on information sets and cor- 
rect, even in the case of counterfactual considerations, 
implies that a player would not, upon encountering an 


unexpected move by another player, revise her beliefs 
about subsequent moves by this or any other player. 

Consider, for instance, the unique subgame-perfect 
equilibrium (BC, XZ) in figure 9.1.1b. The belief of player 
2 at his left node that player 1 would play C at her last 
node already takes into account that player 1 would 
have to have played the unexpected move A in order for 
player 2 to find himself at his left node. 

The whiff of inconsistency accompanying backward 
induction (e.g., Sugden, 1991, pp. 771-774) derives from 
the assumption that players derive their correct beliefs 
about other players’ strategies from the belief that the 
other players are rational—a belief that might turn out 
to be false if an unexpected move occurred. In our pre- 
sentation, however, we do not, and need not, refer to 
beliefs in the rationality of other players. It might be 
argued that such beliefs follow from the assumption that 
players know the extensive form of the game, which, 
after all, contains players’ utilities, whose interpretation 
is tied to RCT. Yet, one need not assume that players 
know other players’ utilities. It suffices to assume instead 
that each player has correct beliefs about the strategies 
played by the other players. This weaker assumption 
avoids any potential inconsistencies from the outset. 

In many games, players’ beliefs about the strategies 
played by the other players are sufficient to determine 
their utility-maximizing moves from the perspective of 
each information set. Not so in the equilibrium (R, R,L) of 
figure 9.1.5b. Player 3’s correct belief that players 1 and 
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1 


Figure 9.1.7 
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The game of Blackmail. The strategic form shows two equilibria, (X, P) and (S,R). Only (X,P) is subgame-perfect. 


2 would both play R implies radical uncertainty for the 
hypothetical situation where, unexpectedly, she finds 
herself called upon to move. In order to decide what she 
would do, player 3 must form beliefs about the alterna- 
tive histories that, counterfactually, might have led to 
this situation. 

Traditionally, RCT resolves radical uncertainty by 
assuming arbitrary beliefs in the form of a subjective 
probability distribution. This is a first step toward fur- 
ther refinements justifying and, often, restricting equi- 
librium beliefs. 

One instructive example of such a refinement is “for- 
ward induction” (Maschler et al., 2013, p. 261). In fig- 
ure 9.1.4, let player 2 know that player 1 would never 
play her—strictly dominated—strategies SD or SM (where 
M=%cU® cD). If player 2 found himself called upon 
to move, he would not believe, then, that player 1 would 
play either D or M: since player 1 must have had played 
S before, D or M would imply the strategies SD or SM, 
respectively, which are ruled out. Therefore, player 2 
would not consider playing either R or N= '4°L@®%oR 
(his best replies to SD and SM, respectively). But then, 
only one subgame-perfect equilibrium remains: (SU, L). 

Equilibria where players are supposed to mix among 
strategies or moves lead to another problem with beliefs 
(cf., e.g., Sugden, 1991, pp. 765-768). In Matching Pennies 
(figure 9.1.3b), game theory predicts the mixed-strategy 
equilibrium (2°U@ %°D, Y%eL ® %okR), while RCT 
says that the players’ behaviors are indeterminate because 
they are indifferent between their pure strategies. This con- 
tradiction is resolved if one assumes that (a) each player’s 
behavior is indeterminate so that the other player faces 
radical uncertainty, and (b) each player assigns subjective 


probabilities 4% to the other player’s choices. Thus, the 
probabilities are interpreted as subjective beliefs without 
physical correlates. Yet, while internally consistent, this 
interpretation clashes with empirical applications where 
the same probabilities appear as physical probabilities of 
actions. 

The same problem occurs in nonstrict pure-strategy 
equilibria like (U, L) in figure 9.1.3d, where player 1 is indif- 
ferent but player 2 nevertheless expects a specific move. 


3.6 Credibility, Commitment, 

and the Explicitness Requirement 

The strategic form represents a game as if players could 
choose strategies. However, as already explained, players 
can choose only moves. A player’s strategy is a collec- 
tion of subjunctive conditionals about her choices at her 
information sets. 

The example of Blackmail (figure 9.1.7) illustrates an 
important consequence of this fact. Considered as such 
a subjunctive conditional, Adam’s strategy R in Blackmail 
is an implicit threat to Eve: “I would play R if you played 
X.” This threat, however, is not credible: Adam would 
not make the move R at this information set. The equilib- 
rium (S, R) in Blackmail assumes that Eve has false beliefs 
about what Adam would do. 

One might wonder whether Adam, to deter Eve, 
could commit himself to report her if she blackmailed 
him. However, in marked contrast to cooperative game 
theory, noncooperative game theory requires games to 
be interpreted under the proviso that only those choices 
exist that are explicitly modeled as possible moves (Har- 
sanyi & Selten, 1988, chapter 1). As it stands, the model 
of Blackmail contains no information set where Adam 
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could choose to commit himself. This leaves us with 
only two modeling options: either the model remains 
as is, expressing our belief that, in the situation we wish 
to model, a commitment option does not exist, or, if we 
believe otherwise, we must explicitly introduce a com- 
mitment option into the model. 

Commitments can take several forms (Gith & 
Kliemt, 2007); for simplicity, we only consider removing 
unwanted options (proverbially, “burning one’s bridges”). 
Consider an extended version of Blackmail (figure 9.1.8) 
where Adam can choose whether to commit himself in 
this sense to report Eve if she blackmails him and where 
Eve can observe whether he commits. In this game, the 
original game of Blackmail appears as a subgame reached 
only if Adam does not commit himself (move N). In the 
subgame reached after committing himself (move C), he 
has no choice between R and P; if Eve chose to blackmail 
him, she would automatically be reported to the police. 

Noncooperative game theory is silent on the question 
of whether, or how, commitment is in fact possible. For 
instance, when revisiting Newcomb’s problem, Spohn 
(2012) allows for (observable) commitment as a purely 
mental process, an exertion of willpower. In a game- 
theoretic model based on this hypothesis, however, such 
a commitment must explicitly appear as a move in the 
game tree (Binmore, 1994, pp. 165-167, 173-182). It is 
one of the merits of the explicitness requirement of non- 
cooperative game theory that it forces us to represent all 
factual assumptions explicitly. 


Figure 9.1.8 

The extended game of Blackmail with initial commitment possi- 
bility for player 2. The only subgame-perfect equilibrium (high- 
lighted) is (CP, XS). 


Max Albert and Hartmut Kliemt 


The explicitness requirement and the assumption 
that preferences are all-inclusive play an important role 
in discussions concerning the Prisoner’s Dilemma (fig- 
ure 9.1.3a). Many contributors found it hard to swallow 
the conclusion that rational players play (D,R) instead 
of (U, L), which both players would prefer. However, in 
order to avoid this conclusion, one must change either 
the utilities or the structure of the game tree such that 
a different game results (Binmore, 1994, pp. 104-109). 


4. Game Theory and “Reasoning about Knowledge” 


The question of how players might come to have correct 
beliefs about other players’ strategies becomes particularly 
relevant if one considers game theory not only as predict- 
ing what players will do but also as advising players what 
they should do in the light of the theory itself. The condi- 
tion that all potential advisees believe in the predictions/ 
prescriptions of a theory and can find no way to do better 
by deviating unilaterally implies equilibrium recommen- 
dations and equilibrium play (Dacey, 1976). 

This insight instigated a tradition of game theory as 
“reasoning about knowledge.” Von Neumann and Mor- 
genstern (2004) already searched for a theory of rational 
play that would imply correct beliefs for rational players 
of unlimited intellectual capacities who know and accept 
the theory. Those who still work in this tradition replace 
knowledge of the theory by “common knowledge” of 
the game tree, that is, all players know the tree, all play- 
ers know that all players know it, and so on, through all 
stages of meta-knowledge. Game theory becomes logical 
reasoning about common knowledge, including common 
knowledge of the theory of rational play (Fagin, Halpern, 
Moses, & Vardi, 1995). 

This logico-philosophical rather than psychological 
“reasoning about knowledge” approach to interactive 
decision making appeals to game theorists (notably, econ- 
omists) to the present day. Identifying equilibrium strate- 
gies by logical reasoning apparently keeps their discipline 
separate from psychology. However, their claim that game 
theory can, in this way, explain human interactions seems 
far-fetched and unlikely to survive the contact with real- 
ity. For this reason, Selten (1999, p. 303), while endorsing 
game theory for its philosophical interest, referred to it as 
“rationology,” likening it to theology rather than science. 

Yet, as our presentation emphasizes, classical game 
theory is not inevitably moored to the “reasoning about 
knowledge” approach and its most recent version, epis- 
temic game theory (see chapter 9.2 by Perea, this hand- 
book). Applications can often rely on simple psychological 
hypotheses about the circumstances under which players 
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may be rational and have correct beliefs about the situa- 
tion and other players’ strategies. Evolutionary game the- 
ory (see chapter 9.3 by Alexander, this handbook) shows 
simple mechanisms by which even players with low cogni- 
tive abilities learn to play sophisticated equilibria. Behav- 
ioral economics (see chapter 9.4 by Dhami & al-Nowaihi, 
this handbook) supplies not only new hypotheses about 
players’ motivations but also alternatives to RCT as the 
basic theory of choice. It is a testimony of its intellectual 
achievement that many of these new developments still 
make use of, or at least take their inspiration from, classi- 
cal game theory. 
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9.2 Epistemic Game Theory 


Andrés Perea 


Summary 


In this chapter, we review some of the most important 
ideas, concepts, and results in epistemic game theory, 
with a focus on the central idea of common belief in ratio- 
nality. We start by showing how belief hierarchies can 
be encoded by means of epistemic models with types 
and how this encoding can be used to formally define 
common belief in rationality. We next indicate how the 
induced choices can be characterized by a recursive elim- 
ination procedure and how the concept relates to Nash 
equilibrium. Finally, we investigate how the idea of com- 
mon belief in rationality can be extended to dynamic 
games by looking at several plausible ways in which 
players may revise their beliefs. 


1. From Classical to Epistemic Game Theory 


Classical game theory, as explored in chapter 9.1 by Albert 
and Kliemt (in this handbook), was pioneered by the sem- 
inal work of von Neumann (1928/1959), von Neumann 
and Morgenstern (1944), and Nash (1950, 1951). It pre- 
sents a series of concepts that select, for every game and 
each of the players in that game, a set of possible choices. 

In epistemic game theory, we concentrate on the beliefs 
that motivate these choices. These may be beliefs about 
the possible choices of the opponents (first-order beliefs) 
but also beliefs about the beliefs of others (higher-order 
beliefs). Putting these first-order and higher-order beliefs 
together leads to belief hierarchies—the language of epis- 
temic game theory. The aim of epistemic game theory is 
to impose reasonable conditions on such belief hierar- 
chies and to explore the behavioral consequences resulting 
from these conditions. 

As with many disciplines in science, it is difficult to 
say when epistemic game theory really started off. Mor- 
genstern (1935/1976), more than 80 years ago, already 
stressed the importance of belief hierarchies in economic 
analysis, but it took a long time before belief hierarchies 
structurally entered the analysis of human behavior in 


economic systems and games. A possible reason for this 
long delay lies in the complexity of a belief hierarchy. 
Despite being a very natural object, it is quite difficult to 
work with because it involves infinitely many layers. 

The purpose of this chapter is to provide an over- 
view of some of the most important ideas and results 
in epistemic game theory, with a focus on the central 
concept of common belief in rationality. The outline is as 
follows: in section 2, we show how infinite belief hier- 
archies in static games can conveniently be encoded by 
means of epistemic models with types and use it in sec- 
tion 3 to formally define common belief in rationality. 
In section 4, we present a recursive elimination proce- 
dure that characterizes the choices that can be rationally 
made under common belief in rationality. In section 5, 
we discuss the epistemic gap between common belief in 
rationality and the famous notion of Nash equilibrium. 
In section 6, finally, we discuss how the idea of common 
belief in rationality can be extended to dynamic games. 

For a more comprehensive overview of epistemic 
game theory, the reader is referred to the overview paper 
by Brandenburger (2007), the textbook by Perea (2012), 
the handbook chapter of Dekel and Siniscalchi (2015), 
the encyclopedia entry by Pacuit and Roy (2015), and the 
book by Battigalli, Friedenberg, and Siniscalchi (in press). 


2. Belief Hierarchies and Types 


The central idea in epistemic game theory is that of com- 
mon belief in rationality. Informally, it states that you 
do not only choose rationally yourself but also believe 
that your opponents will choose rationally, that your 
opponents believe that the other players will choose 
rationally, and so on. Most other reasoning concepts in 
epistemic game theory may be viewed as refinements, or 
variants, of common belief in rationality. The intuitive 
idea of common belief in rationality is already present 
in Spohn (1982) and in the concept of rationalizability 
(Bernheim, 1984; Pearce, 1984), although the latter two 
papers do not formally define the notion. 


544 


An important question is how the idea of common 
belief in rationality can be defined formally. Consider 
a finite static game’ G = (C;,u)ic, Where I is the finite set 
of players, C; the finite set of choices for player i, and 
uj: XjerC; = R player i’s utility function. It is assumed 
that all these ingredients—the set of players, the sets of 
choices, and the utility functions—are commonly believed 
among the players. Moreover, here and in the rest of 
this chapter, we restrict to noncooperative games. When 
we say that player i believes in the opponents’ rational- 
ity, we mean that player i believes that every opponent 
j chooses optimally, given what player i believes that 
player j believes about his opponents’ choices. For this to 
be formally defined, we need to specify i’s belief about j’s 
choice—a first-order belief—together with i’s belief about 
j’s belief about his opponents’ choices, which is a second- 
order belief. Similarly, to formally define that player i 
believes that player j believes in his opponents’ rational- 
ity, we need to additionally specify the belief that i holds 
about the belief that j holds about the belief that every 
opponent k holds about the other players’ choices, which 
is a third-order belief. By continuing in this fashion, we can 
form, for any given player i, an infinite string originat- 
ing in a first-order belief about the opponents’ choices, 
a second-order belief about the opponents’ first-order 
beliefs, a third-order belief about the opponents’ second- 
order beliefs, and so on. Such infinite strings of beliefs are 
called belief hierarchies. They constitute the central con- 
cept of the language of epistemic game theory. 

In view of the fact that belief hierarchies are infinite 
strings, making it hard to write them down explicitly, epis- 
temic game theorists typically encode them in an easy and 
compact way as types in the sense of Harsanyi (1967-1968). 
The main idea is as follows: in a belief hierarchy, player 
i holds, for every opponent j, a belief about j’s choice, j’s 
first-order belief, j’s second-order belief, and so on. That 
is, a belief hierarchy for player i specifies, for every oppo- 
nent j, a belief about j’s choice and j’s belief hierarchy. If we 
replace the words “belief hierarchy” by “type” and formal- 
ize beliefs by probability distributions, we obtain the fol- 
lowing definition of an epistemic model with types: 


Definition 1 (Epistemic Model with Types). Consider a 
finite static game G = (Cj, ui)ier. A finite epistemic model 
for G is a tuple M = (T;,b)ia, where T; is the finite set 
of types for player i, and b;: T; > A(C;x Tj) is i’s belief 
mapping, which assigns to every type t; €T; a probabi- 
listic belief bj(t;)) EA(C; x T;) on the choice-type com- 
binations of i’s opponents. 


In this definition, we have used the following pieces of 
notation: for every finite set X, we denote by A(X) the set 
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of probability distributions on X. By Cx Ti := Xj.i (Gx Th), 
we denote the set of choice-type combinations for i’s 
opponents. 

A finite epistemic model may be viewed as a conve- 
nient way to encode belief hierarchies in a finite man- 
ner. Indeed, for every type in an epistemic model, we 
may derive the full belief hierarchy it induces. 

To see how this works, consider the two-player game 
in table 9.2.1, where player 1’s choices are in the rows 
and player 2’s choices are in the columns, together with 
an epistemic model in table 9.2.2. 

The superscript of a type always specifies the choice 
that is optimal for that particular type. This will be shown 
later. The expression b,(t;) = (0.6): (e,t5) + (0.4)-(f, t3) 
means that type t; assigns probability 0.6 to the event 
that player 2 chooses e and is of type t3, as well as assigns 
probability 0.4 to the event that player 2 chooses fand is 
of type t{. On the other hand, b,(t/') = (g, t}) means that 
type t/ assigns probability 1 to the event that player 2 
chooses g and is of type t3. 

Consider the type t?. As t? believes that, with prob- 
ability 1, player 2 chooses e and is of type t/, the induced 
first-order belief is that player 1 believes that, with prob- 
ability 1, player 2 chooses e. Moreover, as player 2’s type 
t; has the belief (0.6) - (c, t7) + (0.4): (d, t?) about player 
1, player 2’s type t; assigns probability 0.6 to player 1 
choosing c and probability 0.4 to player 1 choosing d. 
Hence, the second-order belief induced by type t? is that 
player 1 assigns probability 1 to the event that player 2 
assigns probability 0.6 to player 1 choosing c and prob- 
ability 0.4 to player 1 choosing d. In a similar fashion, 


Table 9.2.1 
A two-player game 


Nara 
w 
N 


Table 9.2.2 
An epistemic model for the game in table 9.2.1 


T, = {ti tf, ty hy T, = {ty ty, ty} 


b(t!) = (8,62) 

b(t?) = (¢, t3) 

b, (ti) = (0.6) (¢,t;) + (0.4) -(f, #2) 
by (tz) = (0.6) -(c, t;) + (0.4) - (4, t7) 
b,(t3) = (4, t?) 

b, (th) = (c, ty) 


Types 


Beliefs for player 1 


Beliefs for player 2 
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we can derive the higher-order beliefs, and hence the 
full belief hierarchy, for the type t? and for all the other 
types in the epistemic model. 

In the game-theoretic literature, people often use infi- 
nite instead of finite epistemic models, because they wish 
to work with models that encode all possible belief hierar- 
chies. Such exhaustive models are also called terminal type 
structures. That terminal type structures exist for every 
finite static game—something that is far from obvious— 
has been shown by Armbruster and Bdge (1979), Boge and 
Eisele (1979), Mertens and Zamir (1985), and Branden- 
burger and Dekel (1993). For this chapter, we have chosen 
to work with finite epistemic models instead for two rea- 
sons. First, finite epistemic models are easier to work with 
than terminal type structures, since no advanced measure- 
theoretic or topological machinery is needed. Moreover, 
as we will see, this choice does not affect the main results 
we discuss. 

The game-theoretic literature also uses alternative 
ways of encoding belief hierarchies, such as Kripke struc- 
tures (Kripke, 1963) and Aumann structures (Aumann, 
1974, 1976). The first is the predominant model in the 
logical and philosophical literature, whereas the latter is 
often used by economists. Both models use states instead 
of types and assign to every state and every player ia 
choice for player i, together with a belief for player i about 
the states. In a similar way as above, one can then derive 
from such a structure a belief hierarchy for every player 
at every state. In this chapter, we have chosen to encode 
belief hierarchies by means of types, but the whole chap- 
ter could have been written by using Kripke structures or 
Aumann structures instead. 


3. Common Belief in Rationality 


In the previous section, we have seen that belief hierar- 
chies can be encoded by means of epistemic models with 
types. This now enables us to provide a formal definition 
of common belief in rationality, starting from the first 
layer of common belief in rationality, which states that 
player i believes that every opponent chooses rationally. 

To express this event within the formalism of epistemic 
models with types, we must first define what it means for 
a choice to be optimal for a type. Consider an epistemic 
model M = (T;, bier for a static game G = (C;,U)ie, a type 
t;€T;, and a choice c; EC;. Then, 

uj(¢,,t)) = » Dt; (Ci, ti) -Uj(C;, C-;) 

(c_jt_j)€C_jxT_j 

denotes the expected utility for type t; of choosing c;. We 
say that choice c; is optimal for type t; if ui(ci, ti) = ui(ci, t) 
for all c EC;. In the epistemic model of table 9.2.2, it can 
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be verified that a is optimal for the type t/, b is optimal 
for the type t?, and c is optimal for the type t{. Similarly, 
e is optimal for player 2’s type t;, g is optimal for the type 
ts, and h is optimal for the type t?. 

Remember that a type ¢; holds a probabilistic belief 
b(ti) on the opponents’ choice-type combinations. For 
a type t; to believe in the opponents’ rationality means 
that b(t) must only assign positive probability to oppo- 
nents’ choice-type pairs where the choice is optimal for 
the type. 


Definition 2 (Belief in the Opponents’ Rationality). Con- 
sider a finite epistemic model M = (T;, bj); er for a finite 
static game G = (Ci, uij)ier. A type tj ET; believes in the 
opponents’ rationality if b(t) ((c, t)j«i) >O only if, for 
every opponent j i, choice cj is optimal for type f;. 


In the epistemic model of table 9.2.2, it can be veri- 
fied that types t/, t?, t3, and t} believe in the opponents’ 
rationality, but the other two types do not. Indeed, the 
type tj for player 1 assigns positive probability to player 
2’s choice-type pair (f,t§), where f is not optimal for 
the type tz, and hence t; does not believe in player 2’s 
rationality. Similarly, player 2’s type t; assigns positive 
probability to player 1’s choice-type pair (d,t/), where 
dis not optimal for t/, and hence t; does not believe in 
player 1’s rationality. 

With the definition of belief in the opponents’ ratio- 
nality at hand, we can now recursively define k-fold 
belief in rationality for all k= 1, which finally enables us 
to formalize common belief in rationality. 


Definition 3 (Common Belief in Rationality). Consider a 
finite epistemic model M = (T;,Dj)ier for a finite static 
game G = (Ci, Uij)ie. 


(Induction start) A type t,€T; expresses onefold 
belief in rationality if t; believes in the opponents’ 
rationality. 


(Induction step) For k > 1, a type t; ET; expresses k-fold 
belief in rationality if bi(t)((G, {)j.i) > O only if, for 
every opponent j «i, type tf; expresses (k—1)-fold 
belief in rationality. 


A type t; T; expresses common belief in rationality if t; 
expresses k-fold belief in rationality for every k = 1. 


Hence, twofold belief in rationality entails that a type 
only assigns positive probability to opponents’ types that 
express onefold belief in rationality. In other words, the 
player believes that every opponent believes in his oppo- 
nents’ rationality. Similarly, threefold belief in rational- 
ity corresponds to the event that a player believes that 
his opponents believe that their opponents believe in 
their opponents’ rationality, and so on. 
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Within a finite static game G = (Ci, uj)ic, We say that 
player i can rationally choose c;€C; under common belief in 
rationality if there is a finite epistemic model M = (T;, bi)ier 
and a type ¢;€T; such that t; expresses common belief in 
rationality and c; is optimal for t;. That is, choice cj can be 
supported by some belief hierarchy that expresses com- 
mon belief in rationality. 

In the epistemic model of table 9.2.2, it can be veri- 
fied that types t; and t; do not express onefold belief in 
rationality. Indeed, type t; assigns positive probability 
to the choice-type pair (f, tf), where fis not optimal for 
the type t§ and similarly for type t§. Next, types t? and t! 
express onefold but not twofold belief in rationality. To 
see why, note that type t? believes that player 2 is of type 
t;, which does not express onefold belief in rationality, 
and similarly for type t#. Finally, types t/ and t} express 
common belief in rationality. Indeed, type t; believes 
that player 2 is of type t, whereas type t; believes that 
player 1 is of type t/. As both t? and t? express onefold 
belief in rationality, it can inductively be shown that t 
expresses k-fold belief in rationality for all k and hence 
expresses common belief in rationality, and similarly for 
type t'. Consequently, player 1 can rationally choose 
a and player 2 can rationally choose g under common 
belief in rationality. 


4. Recursive Procedure 


Suppose that in a given static game, we are interested in 
the choices that the players can rationally make under 
common belief in rationality. Is there an easy method 
to find these choices, without having to resort to epis- 
temic models with types? That is the question that will 
be addressed in this section. 

The key to answering this question is lemma 3 in 
Pearce (1984), which we will reproduce below. To state 
the lemma formally, we need the following definitions. 
Consider a finite static game G = (Cj, uj)ia, a Choice c;, and 
a belief bj EA(C_;) about the opponents’ choices. Then, 

u,(C;, B)) = > B(C_{) + U,(C, C4) 

ejyeCy 
denotes the expected utility of choice c; under the belief 
b;. Choice c; is called optimal in G for the belief b; if 
ui(ci, bj) = ui(ci, bi) for all ci EC). Choice c; is called strictly 
dominated in G if there is some randomization r; €A(C;) 
such that 

U,(C,C4) < >, A(c)-u,(cf, c.,) for all ci EC. 

cheC; 

In the literature, such randomizations 1; GA(C)) are typ- 
ically called mixed strategies or randomized choices, and they 
are often interpreted as real objects of choice for player i. 
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In this chapter, however, we assume that players do not 
randomize when making a decision, and these random- 
izations r; are merely used as an auxiliary device to charac- 
terize choices that are optimal for some belief. The reason 
is that players are assumed to be expected utility maximiz- 
ers, and hence a player can never increase his expected 
utility by randomizing over his choices. 

Lemma 3 in Pearce (1984) can now be stated as 
follows: 


Lemma 1 (Pearce, 1984). Consider a finite static game 
G = (Cj, u)ier and a choice c; €C;. Then, there is a belief 
b; GA(C_;) such that c; is optimal in G for b; if, and only 
if, c, is not strictly dominated in G. 


This lemma can be used to characterize the choices 
a player can rationally make if he believes in his oppo- 
nents’ rationality. Let G' be the reduced game that 
remains if we eliminate, for every player, the choices that 
are strictly dominated in G. For a player to believe in the 
opponents’ rationality thus means, by lemma 1, that his 
belief is fully concentrated on opponents’ choices in G'. 
By applying lemma 1 to the reduced game G', we thus 
conclude that, for every player, the choices he can ratio- 
nally make if he believes in the opponents’ rationality 
are exactly the choices in G' that are not strictly domi- 
nated in G’. That is, these are the choices that survive two 
rounds of elimination of strictly dominated choices. In a 
similar vein, it can be shown that the choices that can 
rationally be made if a player believes in his opponents’ 
rationality, and believes that his opponents believe in 
their opponents’ rationality (i.e., if he expresses up to 
twofold belief in rationality), are exactly the choices that 
survive three rounds of elimination of strictly dominated 
choices. By continuing in this fashion, we arrive at the 
following elimination procedure: 


Definition 4 (Iterated Elimination of Strictly Dominated 
Choices). Consider a finite static game G = (Ci, Ue 


(Induction start) Let G° :=G be the full game. 

(Induction step) For every k > 0, let G‘ be the reduced 
game that remains if we eliminate from G*" all 
choices that are strictly dominated in G*". 


A choice cj EC; survives iterated elimination of strictly 
dominated choices if c; is in G* for all k > 0. 


By the argument above, we thus see that G’ contains 
exactly those choices that can rationally be made if a 
player believes in the opponents’ rationality. By iterat- 
ing this argument, we conclude that, for every k = 2, the 
k-fold reduced game G* contains exactly those choices 
that can rationally be made under some belief hierarchy 
that expresses up to (k - 1)-fold belief in rationality. This 
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argument already appears in Spohn (1982). In particular, 
the choices that survive the full procedure will be exactly 
those choices that can rationally be made under com- 
mon belief in rationality. This leads to the following cen- 
tral result, which is based on theorems 5.2 and 5.3 in Tan 
and Werlang (1988), and which Brandenburger (2014) 
has called the “fundamental theorem of epistemic game 
theory.” Brandenburger and Dekel (1987) offer in their 
proposition 2.1 a similar result, characterizing common 
belief in rationality by “best reply sets” instead of an 
elimination procedure. 


Theorem 1 (Fundamental Theorem of Epistemic Game 
Theory). Consider a finite static game G = (Cj, uj)ie and 
a choice ¢; €C;. Then, c; can rationally be made under 
common belief in rationality if, and only if, c; survives 
iterated elimination of strictly dominated choices. 


The fundamental theorem would remain unaffected 
if we were to use terminal type structures (hence, infi- 
nite epistemic models) instead of finite epistemic models 
to define common belief in rationality. To illustrate the 
procedure of iterated elimination of strictly dominated 
choices and the theorem above, consider the game G 
from table 9.2.1. In the full game G, it is easily verified 
that player 1’s choice d is strictly dominated by the ran- 
domization that assigns probability 0.5 to his choices a 
and b and that player 2’s choice f is strictly dominated 
by the randomization that assigns probability 0.5 to 
his choices g and h. No other choices are strictly domi- 
nated. Hence, G' is the game obtained after eliminating 
the choices d and f. Within the onefold reduced game 
G', player 1’s choice c is strictly dominated by b (or, 
rather, the randomization that assigns probability 1 to 
b), and player 2’s choice e is strictly dominated by h. 
Hence, G’ is the game obtained from G' after eliminat- 
ing the choices c and e. Finally, within G’, player 1’s 
choice b is strictly dominated by a, and player 2’s choice 
h is strictly dominated by g. As such, only the choices a 
and g survive iterated elimination of strictly dominated 
choices, and hence, by theorem 1, these are the only 
choices that can rationally be made under common 
belief in rationality. 


5. Nash Equilibrium 


For many decades, the concept of Nash equilibrium (Nash, 
1950, 1951) has dominated the classical approach to game 
theory, inspiring many refinements such as perfect equi- 
librium (Selten, 1975) and proper equilibrium (Myerson, 
1978) for static games, as well as subgame-perfect equi- 
librium (Selten, 1965) and sequential equilibrium (Kreps 
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& Wilson, 1982) for dynamic games. See chapter 9.1 by 
Albert and Kliemt (this handbook) for a discussion of the 
latter two concepts. However, for a long time, it remained 
unclear what epistemic conditions are needed for play- 
ers to choose in accordance with Nash equilibrium. The 
purpose of this section is to investigate Nash equilibrium 
from an epistemic perspective and to link it to the con- 
ditions of common belief in rationality that we have 
explored so far. Let us start by giving the definition of 
Nash equilibrium. See also chapter 9.1, where it is called 
mixed-strategy equilibrium. 


Definition 5 (Nash Equilibrium). Consider a finite static 
game G = (Cj, ui)ic. A Nash equilibrium in G is a tuple 
(o)ier Where o; EA(C;) for every player i, such that 
o(ci) > O only if 


> (IT,..06)} U(C,, C4) 2 


C5 = (Cj) je Ci 


YL (Maeien)-ween 


C52 (Cj) ji Ci 


for all c/ €C;. 


In other words, a Nash equilibrium is a tuple of prob- 
ability distributions on choices such that a choice only 
receives positive probability if it is optimal against the 
probability distributions on the opponents’ choices. Tra- 
ditionally, these probability distributions o; have been 
interpreted as conscious randomizations, or mixed strate- 
gies, by the players. A more recent approach, adopted by 
Spohn (1982), Aumann and Brandenburger (1995), and 
other authors, is to interpret o; as the (common) proba- 
bilistic belief that i’s opponents have about i’s choice, 
and this is also the interpretation we use here. 

A Nash equilibrium (oj)i;<er induces, in a natural way, 
a belief hierarchy for player i in which his (first-order) 
belief about the opponents’ choices is given by (gj);.i, his 
(second-order) belief about j’s belief about his opponents’ 
choices is given by (o)x.;, and so on. Such belief hier- 
archies are called simple in Perea (2012). Moreover, this 
belief hierarchy can be shown to express common belief in 
rationality, relying on the optimality conditions in a Nash 
equilibrium. To see this, consider the belief hierarchy for 
player i induced by a Nash equilibrium (o;);~. Then, player 
i only assigns positive probability to a choice c of player j if 
o(c) > 0. By the optimality condition of Nash equilibrium, 
this is only the case if c is optimal against (o,)x.;, which 
is what player i believes that player j believes about his 
opponents’ choices. Altogether, we see that player i only 
assigns positive probability to c; if c is optimal for player 
j, given what player i believes that player j believes about 
his opponents’ choices. That is, with this belief hierarchy, 
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player i believes in j’s rationality. In a similar vein, it can 
be shown that with this belief hierarchy, induced by a 
Nash equilibrium, player i also believes that every oppo- 
nent j believes in his opponents’ rationality, and so on. 
Hence, every Nash equilibrium induces, for every player, a 
belief hierarchy that expresses common belief in rational- 
ity. We can thus say that Nash equilibrium implies com- 
mon belief in rationality. 

But is the other direction also true? Does common 
belief in rationality necessarily lead to Nash equilib- 
rium? The answer, as we will see, is no. Consider the two- 
player game in table 9.2.3, which is similar to the game 
of figure 1 in Bernheim (1984). 

It may be verified that all three choices can rationally 
be made under common belief in rationality. However, 
there is only one Nash equilibrium (0:1, 02) in this game, 
where 0; assigns probability 1 to c and o» assigns prob- 
ability 1 to f. Hence, in this example, Nash equilibrium 
imposes more restrictions than just common belief in 
rationality. But what are these extra restrictions? 

To see this most clearly, consider the epistemic model, 
together with its graphical representation, in figure 9.2.1. 

It may be verified that all types in the epistemic 
model express common belief in rationality. Moreover, 
the respective superscripts of the types indicate the 
choice that is optimal for that type. Remember that only 
the choices c and fare supported by a Nash equilibrium 
in this game. 

Consider the type t/ that supports the choice a—a 
choice that is not supported by a Nash equilibrium. The 
induced belief hierarchy states that, on the one hand, 
player 1 believes that player 2 chooses e but, on the 
other hand, believes that 2 believes that 1 believes that 2 
chooses d. That is, player 1 believes that player 2 is incor- 
rect about 1’s first-order belief. The same can be said about 
his type t?. In contrast, the type t; that supports the Nash 
equilibrium choice c induces a belief hierarchy in which 1 
believes that 2 is correct about 1’s first-order belief. 

It turns out that in two-player games, this correct 
beliefs assumption—that is, that a player believes that 
his opponent is correct about his first-order belief—is 
exactly what separates common belief in rationality 


Table 9.2.3 
A two-player game 
d e f 
a} 0,3 3,0 0,2 
b| 3,0 0,3 0,2 
2,0 2,0 2,2 
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from Nash equilibrium. This is reflected in Spohn’s 
(1982) theorem on page 253, and Aumann and Bran- 
denburger’s (1995) theorem A, which both state that in 
two-player games, mutual belief in rationality, together 
with mutual belief in the actual first-order beliefs, leads 
to Nash equilibrium. Here, mutual belief in rationality 
means that player 1 believes in 2’s rationality, and player 
2 believes in 1’s rationality. Similarly, mutual belief in 
the actual first-order beliefs means that player 1 is cor- 
rect about 2’s first-order belief, and player 2 is correct 
about player 1’s first-order belief. From a one-person 
perspective (in which conditions are imposed on the 
belief hierarchy of a single player i) the Spohn—Aumann- 
Brandenburger conditions thus state that player i 
believes that j is rational, believes that j believes that i 
is rational, that i believes that j is correct about i’s first- 
order belief, and that i believes that j believes that i is 
correct about j’s first-order belief. In particular, Spohn, 
Aumann, and Brandenburger show that the first two lay- 
ers of common belief in rationality, together with the 
correct beliefs assumptions above, are enough to imply 
Nash equilibrium. Not all layers of common belief in 
rationality are needed. Polak (1999) shows, however, 
that if mutual belief in the actual first-order beliefs is 
strengthened to common belief in the actual first-order 
beliefs, then the Spohn—Aumann-Brandenburger condi- 
tions would imply common belief in rationality. Other 
epistemic foundations for Nash equilibrium in two- 
player games, which in some way or another involve the 
correct beliefs assumption above, can be found in Tan 
and Werlang (1988), Brandenburger and Dekel (1989), 
Asheim (2006), and Perea (2007a). As the reasonability 
of the correct beliefs assumption can be debated—after 
all, why should an opponent be correct about your first- 
order belief?—these papers implicitly point at the prob- 
lematic assumptions underlying Nash equilibrium. 

For more than two players, the above conditions are 
no longer enough to characterize Nash equilibrium. For 
such games, Nash equilibrium additionally implies that 
i’s belief about j’s choice must be stochastically indepen- 
dent from i’s belief about k’s choice and that i’s belief 
about j’s belief about k’s choice must be the same as i’s 
belief about k’s choice. The first property follows from 
the fact that in a Nash equilibrium (0;)jc, the belief of 
i about the opponents’ choices is given by the indepen- 
dent probability distributions (o);.i, whereas the second 
condition is implied by the property that i’s belief about 
j’s belief about k’s choice and i’s belief about k’s choice are 
both given by o. These two conditions are not implied by 
common belief in rationality, and hence the gap between 
Nash equilibrium and common belief in rationality is 
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Types | T,= {ttht)}, 7, = {t0.63 

Beliefs for b(t’) = (ety) 
player | 

bi(t)) = (dt?) 

bi(t)) = (ft4) 

Beliefs for | b2(ts) = (at) 
player 2 

bo(t3) = (bt?) 

ba(t3) = (cot) 

Player 1 Player 2 Player 1 
(a,t;’) (dt) ————_» _ (at) 
(6t)) (et;) ———_» _ (ht!) 
(et)), ———> _ (fh) ——————> _ et) 

Figure 9.2.1 


Epistemic model for the game in table 9.2.3 and a graphical 
representation. 


even bigger in games with more than two players. Epis- 
temic foundations for Nash equilibrium in games with 
more than two players can be found in Brandenburger 
and Dekel (1987), Aumann and Brandenburger (1995), 
Perea (2007a), Barelli (2009), and Bach and Tsakas 
(2014). 


6. Dynamic Games 


So far, we have been exploring static games, where all 
players only make one choice, and players choose in com- 
plete ignorance of the opponents’ choices. We now inves- 
tigate how the idea of common belief in rationality can be 
translated to dynamic games. In a dynamic game, players 
may choose one after the other, may choose more than 
once, and may fully or partially observe what the oppo- 
nents have done in the past when it is their turn to move. 
As a consequence, a player may need to revise his belief 
about the opponents when he discovers that his previous 
belief has been contradicted by some of the opponents’ 
past choices. As an illustration, consider the game from 
figure 9.2.2, which is based on Reny (1992). 
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If player 1 believes that player 2 would rationally 
choose g at his last move, then he would choose a at the 
beginning. Common belief in rationality thus seems to 
suggest that player 2 should initially believe that player 
1 chooses a. However, when it is player 2’s turn to move, 
this initial belief has been contradicted by player 1’s past 
play, and hence player 2 must revise his belief about 
player 1. But how? As we will see, there are at least two 
plausible ways for player 2 to revise his belief. 

One option is to interpret player 1’s past move b as 
a mistake, yet at the same time maintain the belief that 
player 1 would choose rationally at his second move, as 
well as the belief that player 1 believes that player 2 would 
rationally choose g at his second move. In that case, player 
2 would believe, upon observing b, that player 1 would 
choose ¢ at this second move, and therefore player 2 would 
choose c. This type of reasoning, in which the players are 
free to interpret “surprising” past moves as mistakes but 
believe that the opponents will choose rationally in the 
future, believe that the opponents always believe that 
their opponents will choose rationally in the future, and 
so on, is called backward induction reasoning and is formally 
captured by the concept of common belief in future rational- 
ity (Perea, 2014). Similar lines of reasoning are present in 
Penta (2015), Baltag, Smets, and Zvesper (2009), and the 
concept of sequential rationalizability (Dekel, Fudenberg, 
& Levine, 1999, 2002; Asheim & Perea, 2005). Backward 
induction reasoning is also implicitly present in the back- 
ward induction procedure (for a survey of the various 
epistemic foundations for backward induction, see Perea, 
2007b) and the equilibrium concepts of subgame-perfect 
equilibrium (Selten, 1965) and sequential equilibrium 
(Kreps & Wilson, 1982; for a formal statement, see Perea 
& Predtetchinski, 2019). 

Another option for player 2, after observing the “sur- 
prising” move PD, is to interpret b as a conscious, optimal 
choice for player 1. However, this is only possible if player 
2 believes that player 1 would choose f afterward and if 
player 2 believes that player 1 assigns a high probability 
to player 2 making the suboptimal choice h at his second 
move. Consequently, player 2 would choose d and, in 
case he is asked to make a second move, choose g. This 
type of reasoning, where a player, whenever possible, 
tries to interpret “surprising” past choices as conscious, 
optimal choices, is called forward induction reasoning. It 
can be formalized by the epistemic condition of strong 
belief in the opponents’ rationality (Battigalli & Siniscalchi, 
2002), which states that a player, whenever possible, 
must believe that his opponents are implementing opti- 
mal strategies.” The concepts that most closely imple- 
ment this type of reasoning are explicable equilibrium 
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3,3 2,2 1,1 


(Reny, 1992) and extensive-form rationalizability (Pearce, 
1984; Battigalli, 1997), where the latter has epistemically 
been characterized by common strong belief in rationality 
in Battigalli and Siniscalchi (2002). See also Battigalli and 
Friedenberg (2012), in which forward induction with 
exogenous restrictions on the players’ beliefs is studied. 

As the example above illustrates, backward and forward 
induction reasoning may lead to different strategy choices. 
Indeed, player 2 chooses c under backward induction rea- 
soning but would choose (d,g) under forward induction 
reasoning. However, both types of reasoning lead to the 
same outcome, which is the terminal history following a. 
Battigalli (1997) has shown that the latter is always true 
in dynamic games with perfect information without rele- 
vant ties by proving that in every such game, the forward 
induction concept of extensive-form rationalizability 
always uniquely leads to the backward induction outcome. 
This result is remarkable, as forward induction and back- 
ward induction represent two completely different lines 
of reasoning. The connection between these two lines of 
reasoning in general dynamic games is one of the many 
intriguing problems in epistemic game theory that need 
further exploration. 


Notes 


1. Such a static game may also correspond to the strategic form 
of a dynamic game, as explained in chapter 9.1 by Albert and 
Kliemt (this handbook). 


2. Strong belief in rationality is very similar to assumption of 
rationality in static games, which has been used by Branden- 
burger, Friedenberg, and Keisler (2008) to epistemically char- 
acterize the iterated elimination of weakly dominated choices. 
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Summary 


Evolutionary game theory analyzes the outcomes of 
interactions between boundedly rational individuals in 
strategic settings. Originally developed in the field of 
population biology, economists and other social scientists 
became interested in evolutionary game theory because 
it provided a natural way of incorporating dynamics into 
game theory. Given the wide variety of different evolu- 
tionary dynamics and models, a number of interesting 
and surprising results have been proven. Under some 
dynamics, both weakly and strictly dominated strategies 
can persist in the limit. And, although it is typically the 
case that population states corresponding to Nash equi- 
libria are fixed points of the evolutionary dynamics, some 
dynamics allow for fixed points that do not correspond 
to Nash equilibria. The connection between evolutionary 
dynamics and the traditional solution concept of game 
theory—the Nash equilibrium—is thus found to be subtle 
and complex. 


1. The Origins of Evolutionary Game Theory 


Evolutionary game theory emerged in the 1970s from the 
realization that frequency-dependent fitness introduced a 
strategic aspect to evolution, even in contexts where none 
of the agents were traditionally “rational.” The first work 
of modern evolutionary game theory is generally under- 
stood to be Maynard Smith and Price (1973), in which the 
concept of the evolutionarily stable strategy (ESS) was intro- 
duced.' Yet one can see the beginnings of game-theoretic 
approaches to the study of evolution in the work of R. A. 
Fisher, who, in The Genetical Theory of Natural Selection 
(1930), attempted to explain the approximate equality 
of the mammalian sex ratio using arguments that were 
essentially game-theoretic in nature. 

After its introduction in population biology, evolution- 
ary game theory became of interest to economists and 
social scientists more generally. Initially, it was hoped that, 
because the concept of an evolutionarily stable strategy 


provided a proper strengthening of the Nash equilibrium 
solution concept, evolutionary game theory could offer 
insight into the equilibrium selection problem: when 
multiple Nash equilibria exist, which one rational players 
would settle upon. In addition, the fact that evolutionary 
game theory provided dynamical models of quasi-rational 
deliberation was seen to address a lacuna in the traditional 
theory of games, which largely provided a static analysis 
of strategic problems. In Theory of Games and Economic 
Behavior, von Neumann and Morgenstern wrote, 


We repeat most emphatically that our theory is thoroughly 
static. A dynamic theory would unquestionably be more 
complete and therefore preferable. But there is ample evi- 
dence from other branches of science that it is futile to try to 
build one as long as the static side is not thoroughly under- 
stood. (§ 4.8.2) 


The “most [emphatic]” reminder of the von Neumann 
and Morgenstern theory being static was needed because, 
although the extensive-form representation of games 
does make explicit the temporal relation between the 
choice nodes of players, individual strategies were repre- 
sented atemporally, where a strategy specified the choice 
of a player, in advance, at all possible choice nodes. 

Over time, it was recognized that evolutionary game 
theory was less successful in delivering on these aims than 
initially hoped. The equilibrium selection problem reap- 
peared in the form of competing notions of evolutionary 
stability, and the plurality of dynamical models yielded 
varying results regarding the long-term survival of strate- 
gies. For example, whereas some concepts of evolution- 
ary stability rule out even weakly dominated strategies, in 
the dynamical setting, it turns out that both weakly and 
strictly dominated strategies can persist for some families 
of dynamics. In addition, some families of dynamics— 
particularly imitative dynamics—have the property that 
not every fixed point of the dynamics corresponds to a 
Nash equilibrium of the underlying game. 

Since its inception, evolutionary game theory has 
morphed into a more general study of the population 
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dynamics of boundedly rational agents (see chapter 8.5 
by Hertwig & Kozyreva, this handbook), with the once- 
central concept of an evolutionarily stable strategy, and its 
subsequent refinements, no longer in central place. The 
family of models developed provides useful mathemati- 
cal tools for modeling and understanding the behavior of 
quasi-rational individuals interacting in strategic contexts, 
but care needs to be exercised to ensure that the model and 
the target system are aligned before drawing conclusions. 


2. Game-Theoretic Fundamentals 


Consider a game played by a group of N individuals, 
where each player i has a set S; of pure strategies. Each 
strategy s ES; can be thought of as an action available to 
player i; the complete set of actions taken by all players 
determines the payoffs each player receives. In traditional 
game theory, it often proves useful to allow a player to 
choose a strategy at random, so as to keep other individu- 
als uncertain as to what she will do; such strategies are 
called mixed strategies and correspond to probability dis- 
tributions over the set of pure strategies. The set of mixed 
strategies for player i is denoted Aj, and a particular mixed 
strategy for player i will be denoted o; €A;. Mixed strat- 
egies, while indispensable for many important theoreti- 
cal results in traditional game theory (see chapter 9.1 by 
Albert & Kliemt, this handbook), such as Nash’s theorem 
that every finite n-player game has at least one equilib- 
rium, play a less central role in evolutionary game theory. 
If ¢ =(o,,...,O6y) denotes the particular strategies 
used by each player, also known as a strategy profile, the 
payoff to player i is given by a(G), where a; is the pay- 
off function for that player. In two-player games, we will 
use the notation “z(o|)” to denote the fitness of player i 
when ois played against u. A two-player game is symmetric 
when, for all strategies o, u, the payoff of o played against 
u does not depend on the identity of the player; for such 
games, the payoff function will simply be written “z.” 
The fundamental solution concept for traditional 
game theory is the Nash equilibrium, defined as follows: 


Definition. A strategy profile o*=(0;,...,0°) is a 
Nash equilibrium if and only if for each player i and 
all strategies oj EAi, 


Tj (07, o*) 27(0;, o*), 


where “(o;, 0*,)” denotes the strategy profile identical 
to o* except that the strategy for player i is replaced 
by oi. 


A Nash equilibrium is a set of mutual best responses for 
each player, where no individual can improve their payoff 
by adopting an alternative strategy. This property makes 
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it a natural minimal solution concept for traditional 
game theory. All games with a finite number of play- 
ers and finitely many strategies have at least one Nash 
equilibrium. 


3. Evolutionarily Stable Strategies 


To see that the concept of a Nash equilibrium is too weak 
for capturing the idea of evolutionary stability, consider 
the game shown in figure 9.3.1a. That game has two Nash 
equilibria in pure strategies: (S;,5;) and (S2, $2). But imag- 
ine this game played in a population where individuals 
are paired at random and fitness corresponds to expected 
payoff. Informally, it is clear that a population consisting 
entirely of the S, type would not be evolutionarily stable. 
In such a population, a novel S; mutant receives the same 
payoff as the incumbent population and thus does not 
incur a fitness disadvantage. This means an S, mutant 
can invade the population. If two S$; mutants appear, the 
payoff of an S;-S, interaction exceeds that of an S:-S2 
interaction, which means the S, type has an expected fit- 
ness greater than the average of the population and thus 
would be expected to increase in number, potentially 
even driving the S, type to extinction.” The problem 
stems from the fact that a Nash equilibrium only requires 
that deviant play does no better than the equilibrium strat- 
egy, while allowing for the possibility that deviant play 
may do as well as the equilibrium strategy. 

A strict Nash equilibrium is one where any deviation 
from the equilibrium profile leaves players worse off. 
To see that the concept of a strict Nash equilibrium is 
too strong to capture the idea of evolutionary stability, 
consider the game in figure 9.3.1b, the Hawk—Dove of 
Maynard Smith and Price (1973).* In this game, two indi- 
viduals compete over a resource with value V. The strate- 
gies individuals may adopt are “Hawk,” which escalates 
to the point of fighting, if necessary, and “Dove,” which 
makes a show of escalating initially but then backs down 
just before the point of fighting. Given this, if a Hawk 
meets a Dove, the Hawk gets the entire resource. If two 
Doves meet, they share the resource. If two Hawks meet, 
they escalate to the point of fighting until one is injured, 
with cost -C, and retreats.* If V<C, this game has no 
pure-strategy Nash equilibrium and one mixed-strategy 
Nash equilibrium where both individuals play Hawk 
with probability V/C. Denote this mixed strategy by o*. 
From the fundamental theorem of mixed-strategy Nash 
equilibria (see Gintis, 2009), it can be shown that 


m(Hawk | 0*) = a(Dove|o*) = a(o*| 0%). 


Therefore, for any other mixed strategy u = o*, it is 
the case that a(u|o*) = a(o* | o*), and so o* is not a strict 
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Hawk Dove 
V- V- 
Si Sy Hawk ( _ <| (V, 0) 
Si (2, 2) d, 1) 
VV 
S| a.) (1,1) Be) (3 ¥] 


(a) A Nash equilibrium allows for 
evolutionary drift, and hence may 
not be evolutionarily stable. 


Nash equilibrium strategy. However, unlike the game in 
figure 9.3.1a, here it can be shown that 2(o0* | uw) > a(u| uw). 
This means that the mutant strategy wu is at a fitness dis- 
advantage in that it does less well when it plays against 
itself than the incumbent strategy o* does when it plays 
against uw. This means that the mutant strategy u would 
be unable to invade and, hence, the incumbent strategy 
ois evolutionarily stable. 

These considerations led Maynard Smith (1982) to 
propose the following: 


Definition. A strategy o is an evolutionarily stable strat- 
egy (ESS) if and only if for all strategies u = o, it is the 
case that 


either a(0|o) > a(u|o), 
or 2(o| 0) = (ul o) and x(o| yu) > a(ul|u). 


The first clause states that every strict Nash equilibrium 
strategy is an ESS; the second clause states that, if an ESS 
is not a strict best reply to itself, then the ESS must be a 
strict best reply to all other strategies. Although it is true 
that all strict Nash equilibrium strategies are evolution- 
arily stable, the converse does not hold. The second- 
order best-reply condition means that an ESS provides 
a real strengthening of the Nash equilibrium solution 
concept. 

An alternative definition of an evolutionarily stable 
strategy, first introduced by Taylor and Jonker (1978) 
and used by Weibull (1995) and others, is as follows. Let 
oand u be strategies, and let eu + (1- €)o for e > 0 denote 
the strategy, which « of the time plays u and 1 -« of the 
time plays o. For small ¢, this can be thought of as either 
a small “tremble” around the base strategy o or, equiva- 
lently, as the strategy employed by a randomly chosen 
opponent in a population of individuals where almost 
all play o but a small number of mutants adopt u. 


Definition. A strategy o is an evolutionarily stable strat- 
egy if and only if for every strategy u = o, there exists 
a0Q<e, <1 such that 


mo|eut (1-¢)o) > a(uleut (1-e)o) 


for all O<€< &,. 


(b) A game whose Nash equilibrium 
is not strict when V< C, yet is 
evolutionarily stable. 


Figure 9.3.1 
Why the concept of a Nash equilibrium is 
too weak to capture the idea of evolution- 
ary stability. 


It is easily shown that this definition, which can be inter- 
preted as a mathematically precise formulation of the 
definition given by Maynard Smith and Price (1973), is 
equivalent to the one proposed by Maynard Smith (1982). 

The above definition of an ESS states that the expected 
payoff of an ESS in a mixed population consisting of the 
ESS and a small number of mutants is strictly greater than 
the expected payoff of the mutant in that same mixed 
population. However, Hofbauer, Schuster, and Sigmund 
(1979) give the following characterization of an ESS, which 
not only has the benefit of being conceptually clearer but 
also proves to be more useful when we move to consider- 
ing evolutionary dynamics in the next section: 


Definition. A strategy ois locally superior if there exists 
an e-neighborhood around o such that, for all strate- 
gies uw # o in that neighborhood, it is the case that 
mo| tu) > mul pu). 

From this, one can then prove the following: 


Theorem 1. A strategy o is an ESS if and only if o is 
locally superior. 


There are a number of theorems one can prove about 
ESSs. For example: 


Theorem 2. No weakly dominated strategy is an ESS. 


This is of interest because, as shown in figure 9.3.2, some 
weakly dominated strategies constitute Pareto-optimal 
Nash equilibria. In such cases, it can be rational to play 
a weakly dominated strategy even though it is not evo- 
lutionarily stable. 

The support of a mixed strategy o is the set of pure 
strategies to which o assigns nonzero probability. One 
can show that if uw and o are ESSs with identical sup- 
ports, then u = o. From this, it immediately follows that 


Si S2 
Si (1, 1) (100, 0) 
S| (0, 100) (100, 100) 


Figure 9.3.2 
The fact that no weakly dominated strategy is an ESS means that 
some Pareto-optimal Nash equilibria are excluded, as shown here. 
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Rock Paper Scissors 
Rock (0, 0) (-1, 1) d,-1) 
Pape |) (=) (0, 0) C1, 1) 
Scissors} (1,1) | (1,-1) (0, 0) 
Figure 9.3.3 


The game of Rock—Paper-Scissors, which has no ESS. 


Theorem 3. The number of ESSs is finite (and possibly 
zero). 


The latter half of the above theorem is easily proven 
by considering the well-known game of Rock—Paper- 
Scissors, shown in figure 9.3.3. The only Nash equilib- 
rium is in mixed strategies and assigns equal probability 
to all three pure strategies. The zero-sum nature of the 
game means that neither of the two conditions in the 
definition of an ESS are met, and so this game has no 
evolutionarily stable strategy. 


4. Dynamics for Evolutionary Games 


The main shortcoming of the evolutionarily-stable-strategy 
concept is that it, like that of a Nash equilibrium, is static. 
It would be useful if we could say something about the 
behavior of a population in an out-of-equilibrium envi- 
ronment, especially in games where multiple ESSs exist 
or where no ESS exists. For this reason, we now turn to 
dynamical models of evolutionary games. As we shall see, 
there is an imperfect match between static and dynami- 
cal notions of evolutionary stability, and although there 
are some stability concepts that possess a useful degree of 
generality, considerable variation exists in the behavior of 
a population between continuous and discrete models, as 
well as between unstructured and structured population 
models. 

To begin, let S = {S;,..., S,} denote the set of pure 
strategies and let pt) €[0, 1] denote the proportion of 
the population using strategy i at time ¢. It is assumed 
that no player employs a mixed strategy and that 
Yi-1 p(t) =1. Let A, denote the set of all possible fre- 
quency distributions of strategies over the population. If 
the only thing that matters, from an evolutionary point 
of view, is the frequency with which a strategy is used 
in the population, then f(t) = (p,(t),...,p,(t))eA, 
represents the total state of the population at time t. A 
game, in this context, is a function W: A, = R", which 


dp, _ Rate at which the population 


dt switches to strategy i 
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maps states of the population to a vector of expected 
payoffs—which will typically be thought of as expected 
fitnesses—for each pure strategy. In what follows, we use 
W, (p(t)) to denote the expected payoff to strategy i when 
the population is in state p at time ¢. 

Sandholm (2010a, 2010b) provides a useful frame- 
work for thinking about evolutionary dynamics, dem- 
onstrating how continuous dynamics at the population 
level can be derived from particular learning rules—also 
known as revision protocols—used by the individuals 
comprising the population. This in effect provides the 
“microfoundations for existing dynamics” (Sandholm, 
2010b, p. 31), the evolutionary game theory analogue of 
providing microfoundations for macroeconomics. 

Sandholm models a revision protocol as a map from the 
current expected payoffs of each strategy in the population, 
as well as its present composition, to a transition matrix 
expressing the rate at which strategies switch from one type 
to another. More formally, a revision protocol is a map, 
generally assumed to be Lipschitz-continuous, as follows: 


p:R"xA,7> RY". 


Given a particular vector of expected payoffs 7€R” and 
a population state peA,, the ijth entry of p(z, p), writ- 
ten p,(Z, Pp), is the rate at which individuals switch from 
strategy i to strategy j. 

Once a revision protocol is specified, the evolutionary 
dynamics for the population are derived by substituting 
the revision protocol into the following schema of differ- 
ential equations defined on A,, (see equation 1 below). For 
the precise mathematical details, see Sandholm (2010b). 

Much of the early literature in evolutionary game the- 
ory focused on the replicator dynamics, the first dynami- 
cal model of evolutionary games, proposed by Taylor and 
Jonker (1978). The core idea behind the replicator dynam- 
ics is that the proportion of the population that follows a 
given strategy increases if the fitness of that strategy is 
higher than the average fitness of the population, which 
is denoted by @(P). The average fitness of the population 
is just the weighted average of the individual-strategy fit- 
nesses, with the proportion of the population following 
the respective strategies used as weights: 

OB) = Dp Wi(B). 

ieS 
The continuous replicator dynamics is the system of differ- 
ential equations given by 


| | Rate at which the population (1) 


switches from strategy i 
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Ps = p,-1W)- BY, fortes. 
Note that if p;,(0) > 0, then p,(é) > 0 for all t. This is of course 
compatible with p(t) — 0 as t — ~; the point is that, under 
the replicator dynamics, strategies can be driven to extinc- 
tion in the limit, but they cannot actually be driven out in 
finite time. A related point is that the replicator dynamics 
cannot introduce strategies into the population if they are 
originally absent: if p(O) = 0, then p,(t) = 0 for all t. 

In the simplest case, if population members inter- 
act at random and the probability of interacting with 
strategy j equals p(t), then the fitness of strategy i is 
W,(p(t)) = 2p, (t)-2(S;|S;). This is the continuous Jinear 
replicator dynamics (henceforth, simply “the replicator 
dynamics”). 

Schlag (1998) showed that the replicator dynamics can 
be derived from the following revision protocol: suppose 
that each player in the population selects someone at ran- 
dom and compares their individual payoff with that of the 
person selected. If the person selected received a higher 
payoff, then the player adopts the strategy of the person 
selected with a probability proportional to the difference in 
payoffs. That is, if # denotes the vector of expected payoffs 
for the population in state P, then p; (z, P) =p, [4-7]. 
where [x], = max(x, 0). The replicator dynamics, then, is the 
population-level description of the outcome when people 
imitate with probability proportional to success. 

At this point, anumber of interesting differences already 
emerge between the static ESS concept and the outcomes 


Si S> S3 
a) (1,1) (1, 0) 
So} (1, 1) (1, 1) (0, 0) 
Ss} (0, 1) (0, 0) (0, 0) 
(a) A game where strategy S2 is weakly 
dominated by S\. 
Figure 9.3.4 


557 


of dynamical models: no weakly dominated strategy can 
be an ESS, but the replicator dynamics does not necessar- 
ily eliminate weakly dominated strategies. Indeed, in the 
case of the game of figure 9.3.4, almost all evolutionary 
trajectories converge to a state in which some proportion 
of the population follows a weakly dominated strategy. 

Although the replicator dynamics need not eliminate 
weakly dominated strategies, Akin (1980) showed that, 
in most cases, the replicator dynamics will eliminate 
strictly dominated strategies: 


Theorem 4. Let Pp € A, be a point in the interior of the 
simplex. If S;is strictly dominated, then lim,_,..p,(t) = 0 
under the replicator dynamics. 


The requirement that all strategies must be represented 
in the initial state is easily seen to be necessary: consider 
the prisoner’s dilemma, and suppose that the population 
begins in the state where everyone follows the strategy 
Cooperate. Although Cooperate is strictly dominated 
by Defect, the fact that the replicator dynamics cannot 
introduce absent strategies means that the strictly domi- 
nated state of All-Cooperate will endure. 

Although the replicator dynamics generally eliminates 
strictly dominated strategies, the same is not true for 
other types of evolutionary dynamics. Indeed, Hofbauer 
and Sandholm (2011) show that many families of evo- 
lutionary dynamics have the property of not eliminat- 
ing strictly dominated strategies. This result shows how 
classical conceptions of rational behavior, at the individ- 
ual level, can become decoupled from population-level 


(b) In almost all cases, the replicator dynamics converges 
to a polymorphic state featuring a mix of both S; and Sp. 


Weakly dominated strategies may not be eliminated by the replicator dynamics. Shading indicates the relative speed of the 


evolutionary trajectories. 
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outcomes. In what follows, two additional evolutionary 
dynamics are introduced, which—despite the plausible 
nature of the underlying revision protocol—do not, in 
general, eliminate strictly dominated strategies. 

As we've seen, the replicator dynamics can be derived 
from a revision protocol that imitates strategies with 
probability proportional to success. One might wonder 
whether imitation is a reliable guide to future payoffs. 
Imitation certainly suffers from an inability to introduce 
new strategies into the population. Perhaps it would be 
better to adopt a strategy only taking into account its 
performance compared to aggregate properties of the 
overall population. One such revision protocol, which 
yields an interesting evolutionary dynamic, is the fol- 
lowing: suppose that the rate at which players switch 
to strategy j does not depend on what strategy they cur- 
rently employ, or what their expected payoff is in the 
current population state, but rather only on whether the 
expected payoff of strategy j exceeds the average payoff 
of the population. Sandholm (2010a) shows that when 


p;(W(p), P) = [W,(P)-9(p)]., forall jes, 


the evolutionary dynamic obtained is the Brown—Nash- 
von Neumann (BNN) dynamic (Brown & von Neumann, 
1950). This dynamic belongs to a more general class of 
dynamics known as separable excess payoff dynamics. The 
BNN dynamic takes the following form: 


. — 1w,(B)- oP), — 1 Y[W;,(B)- 4B), for allies. 


jeS 


dt 


Rock Paper Scissors 

Rock (0, 0) (-2, 1) di, —2) 
Paper] (1,-2) (0, 0) (-2, 1) 
Scissors] (—2, 1) (1, -2) (0, 0) 


(a) The game Bad Rock—Paper-Scissors. 


Figure 9.3.5 
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One interesting difference between the BNN dynamic 
and the replicator dynamic is that the BNN dynamic can 
introduce new strategies into the population. Figure 9.3.5 
shows this for the game Bad Rock—Paper-Scissors. The 
phase diagram shows the evolutionary trajectories for 
three different initial population states, each beginning 
at vertices of the simplex. The BNN dynamic moves the 
population along each of the respective faces in the direc- 
tion of the dominating strategy. At some point, there is 
a fitness advantage to playing the third, unrepresented, 
strategy, and so the dynamics moves into the interior of 
the simplex and converges to an orbit. 

The revision protocol that yields the BNN dynamics 
compares alternative strategies to the average payoff of 
the population. In a sense, this is a strange motivation 
because the average payoff of the population, in most 
cases, does not correspond to the payoff obtained by 
any particular strategy. As an alternative, consider the 
revision protocol that selects a new strategy by compar- 
ing the expected payout of alternative strategies to the 
expected payout of one’s current strategy, where only 
those strategies with higher expected payoffs have a 
nonzero chance of being adopted. That is, 


pi(W (B, B)) = [W, (B)- Wi (PIL. 
This revision protocol generates 
dynamic first discussed by Smith (1984) and hence is 


known as the Smith dynamic. This dynamic is a mem- 
ber of a more general family of evolutionary dynamics 


an evolutionary 


1.41 


The Brown—Nash-von Neumann dynamic can introduce new strategies into the population that are not initially present. 


Evolutionary Game Theory 


known as impartial pairwise comparison dynamics. (It is 
“impartial” in that the switch rate does not depend 
upon the player’s current strategy i.) The Smith dynamic 
takes the following form: 
dp; a > > > 
ie dP [W,(B)- W (B)I. ~ PRIM B)- WB. 
Both the Smith dynamic and the BNN dynamic allow 
strictly dominated strategies to persist in the limit. To 
see this, consider the game of figure 9.3.6a. Here, the 
game of Bad Rock—Paper-Scissors is augmented with a 
fourth strategy: a “feeble twin.” The feeble twin strategy 
is essentially the same as Scissors except that all of its 
payoffs are decremented by an additional amount «> 0. 
Figure 9.3.6b shows how, under the Smith dynamic, 
populations beginning at the four vertices of the simplex 
(states such as All-Rock, All-Paper, etc.) follow trajecto- 
ries such that all four strategies are played with positive 
probability, including the strictly dominated feeble twin 
strategy. 
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The phenomenon illustrated in figure 9.3.6 is a spe- 
cific instance of the following general result: 


Theorem 5 (Hofbauer & Sandholm, 2011). For all evolu- 
tionary dynamics that belong to either the family of 
impartial pairwise comparison dynamics or the family 
of separable excess payoff dynamics, there exists a 
game such that, from most initial conditions, there is 
a strictly dominated strategy played by a proportion 
of the population that is bounded away from 0 and 
exceeds % infinitely often as time approaches infinity. 


One interesting point about the result of theorem 5 is 
that both the BNN dynamics and the Smith dynamics are 
generated by revision protocols that are not obviously 
implausible. Thus, irrational outcomes at the population 
level—the survival of strictly dominated strategies—can 
be generated by revision protocols that, at the individual 
level, may appear sensible. 

All of the above three dynamics are single-population 
models, in that they assume the underlying game is 


Rock Paper Scissors Twin 
Rock (0, 0) (-2, 1) (1, -2) (1, -2-«) 
Paper (i, -2) (0, 0) (-2, 1) (2, 1-€) 
Scissors (-2, 1) d,-1l) (0, 0) (0, —€) 
Twin | (-2-€, 1) (i -e,-1) (-e, 0) (-e€,-€) 


(a) In this version of Bad Rock—Paper-Scissors, the fourth strategy Twin is the 


same as Scissors except that all of its payoffs are decreased by €. 


(b) Preservation of the strictly dominated Twin strategy. 


Figure 9.3.6 
7 The Smith dynamic allows strictly dominated 
strategies to be preserved. 
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symmetric. Since there is no way of distinguishing row and 
column players, it must be the case that m(S;|5Sj) = 2(S;| Si). 
Asymmetric games are typically analyzed using multipopu- 
lation models, one population for Row and another pop- 
ulation for Column, which give rise to a coupled system 
of differential equations. Suppose, for example, that the 
base game is the two-player asymmetric game Battle of the 
Sexes, shown in figure 9.3.7a. The two-population replica- 
tor dynamics treats the Row and Column players as being 
drawn from separate subpopulations, where the expected 
fitness of a Row player depends on the distribution of strat- 
egies found in the Column subpopulation, and vice versa, 
as indicated in figure 9.3.7b. (The superscripts R and C 


Row 
Column 
Si Sy 
Si dj, 2) (0, 0) 
So (0, 0) (2, 1) 


(a) Battle of the Sexes. 


R oC 
Pr Pz 
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denote the Row and Column subpopulation, respectively.) 
In this case, the replicator dynamics takes the following 
form (again, the explicit time dependencies are suppressed 
for clarity): 


dpk R R(RC DR * 

wee - [WR (po) - o(p*)], fori=1,2, 
and 

dpe = - : 

a = pr [Wo(B*)- oB°)], fori=1,2. 


For the payoff matrix given in figure 9.3.7a, the specific 
system of coupled differential equations is obtained by 
substituting the following values into the above: 


Column 


(b) A two-population model with the 
interaction probabilities indicated. 


Si 


So 


(c) Phase diagram for the two-population, continuous linear 


replicator dynamics for Battle of the Sexes. The two stable rest 
points are (1, 0), (1, 0)) and ((0, 1), (0, 1)), denoted by solid circles; 


unstable rest points are indicated by unfilled circles. 


Figure 9.3.7 
A two-population version of the replicator 
dynamics. 
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WE(P°) = pe, 
WE(p°) = 2 pS, 
GP") = pEpe + 2pRps, 


WE (p*) = 2pk, 
WE (p*) = Ps, 
O(p°) = 2pRpE + pRps. 


Figure 9.3.7c illustrates the phase diagram for Battle of 
the Sexes under the two-population replicator dynam- 
ics. Cressman (2003) provides a detailed investigation 
of multipopulation models with reference to extensive- 
form games. 


5. Stability 


One natural question to ask is what connection—if any— 
exists between rest points of the various dynamical sys- 
tems under consideration and the Nash equilibria (and/or 
evolutionarily stable strategies) of the underlying game. As 
a preliminary, an important conceptual distinction needs 
to be made: the various dynamics considered model the 
evolutionary flows over population states, where a popu- 
lation state is a distribution of strategies over the individual 
players. In contrast, a Nash equilibrium strategy (or an 
evolutionarily stable strategy) is a probability distribution 
over pure strategies adopted by a player. These are very 
different entities. But, as we will see, as long as we are 
careful about the conceptual differences, important con- 
nections exist. 

Inspection of figure 9.3.7c suggests one possible con- 
nection. The underlying game has three Nash equilibria: 
((1,0), (1,0)), (0,1), (0,1)), and (4,%), (4,14). Of the three 
Nash equilibria, the first two feature evolutionarily stable 
strategies. If all three of these Nash equilibria are inter- 
preted as population states, all correspond to rest points 
of the dynamics, with the two states corresponding to 
evolutionarily stable strategies being stable points of 
the dynamics. (The exact meaning of “stability” will be 
specified below.) However, the fact that additional rest 
points exist shows that, at least for the replicator dynam- 
ics, it is not the case that the rest points correspond to 
Nash equilibria.’ A precise statement of the relationship 
between the two concepts requires introducing some 
definitions from the theory of dynamical systems. 

A rest point of a dynamical system (also known as a fixed 
point) is a point p such that dp,/dt = 0 for alli=1,..., 70. 
A stability concept provides a characterization of the 
behavior of a dynamical system in the local region around 
a fixed point. There are several natural stability concepts 
used in evolutionary game theory. 


Definition. Let Pe A, be a rest point of the replica- 
tor dynamics. Then fp is said to be Lyapunov-stable 
if for every «>0, there exists a 6>0 such that 
| p(O) — p| < 6 implies | p(t) - p| < € for all t=0. That is, 
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every trajectory that is sufficiently close to the fixed 
point p (ie., no more than 6 away) remains close to P 
(i.e., no more than « away) for all future times. 


The intuition behind Lyapunov stability is that there 
is not a local “push” away from the rest point. Another 
intuition about stability is that there is a local “pull” 
toward the rest point. This motivates the following: 


Definition. Let Pp be a fixed point of the replica- 
tor dynamics. Then p is asymptotically stable if p is 
Lyapunoy-stable and, in addition, there exists a 6>0 
such that if |p(0) — p| <6, then lim, ,.. p(t) = p. 


Asymptotic stability captures one aspect of evolution- 
ary stability: that the population is not easily displaced 
from the rest point corresponding to a particular distri- 
bution of strategies. A displacement from an asymptoti- 
cally stable rest point, provided it is not too great, will 
return to the rest point in the limit. 


Definition. If p is Lyapunov-stable but not asymptoti- 
cally stable, then P is said to be neutrally stable. 


In light of the Hofbauer et al. (1979) result that an 
evolutionarily stable strategy is one that is locally supe- 
rior, we can introduce an analogous definition in the 
context of evolutionary games. In particular, Hofbauer 
and Sigmund (2002) propose the following: 


Definition. Let pe A,, and let A be the payoff matrix 
for the underlying evolutionary game. Then f is a 
Nash equilibrium if, for all 7 € A,,, it is the case that 


p-Ap=q-Ap. 


The state p is an evolutionarily stable state if, for all 
>>. : > 
q # p ina neighborhood around p, 


p-Aq2=q-Aq. 


Note that, despite the mathematical similarity 
between the various definitions, the term “evolution- 
arily stable state” is used to stress the difference between 
the population and the strategy interpretation. But, 
in effect, an evolutionarily stable state is the distribu- 
tion corresponding to an evolutionarily stable strategy 
interpreted as the frequency with which pure strategies 
appear in a population. For example, in the case of the 
Hawk-Dove game, the evolutionarily stable strategy is 
the mixed strategy o =“ Hawk + (1- “%) Dove, and the 
evolutionarily stable state is the population where “% of 
the population follows Hawk and 1- % follows Dove. 

Returning to the game of Rock—Paper-Scissors from 
figure 9.3.3, the phase diagram for the replicator dynam- 
ics is shown in figure 9.3.8. The rest point corresponding 
to the single Nash equilibrium of the game, (%,%,\), 
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although not an evolutionarily stable state, is neutrally 
stable. Any small displacement from the rest point in 
the center of the simplex will result in the population 
cycling around the rest point in a stable orbit. 

These two observations provide an illustration of the fol- 
lowing theorem, proven by Hofbauer and Sigmund (2002): 


Theorem 6. Let A be the payoff matrix for a symmetric 
two-player game. Then 


if pe A, is a Nash equilibrium of A, then f is a rest 
point of the replicator dynamics; 


if p is Lyapunov-stable under the replicator dynamics, 
then it is a Nash equilibrium of the underlying game. 


One can also prove (see Hofbauer & Sigmund, 2002) that the 
following connection holds between evolutionarily stable 
states and asymptotic stability, for the replicator dynamics: 


Theorem 7. Let A be the payoff matrix for a symmetric 
two-player game. If peA,, is an evolutionarily stable 
state for A, then p is asymptotically stable under the 
replicator dynamics with A as the underlying game. 


It turns out that if one adopts a slightly stronger 
stability concept than that of an evolutionarily stable 
state, the above result for the replicator dynamics can be 
extended to multiple families of dynamics. Taylor and 
Jonker (1978) define a regular ESS as follows: 

Definition. Let S = {1,..., n} be a set of pure strategies 

for a symmetric two-player game with payoff matrix A. 

Let obea Nash equilibrium for A. Then ois a regular ESS 

if a(i| 0) <a(o|o) whenever i¢ supp(o), and ¥-AX¥ <0 


whenever supp(x) Csupp(o), X# 6 and =x; = 0. 


The concept of a regular ESS is a slight strengthening 
of the ESS concept. Every regular ESS is an ESS, but not 
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conversely. However, Taylor and Jonker note that most 
ESSs are regular, and if a is a payoff function for which 
a nonregular ESS exists, one can find another payoff 
function x’, arbitrarily close to a, that only has regular 
ESSs. Intuitively, a regular ESS is an ESS o where the sup- 
port of o contains all of the pure strategies that are best 
responses to o. 
One can then prove the following: 


Theorem 8 (Hofbauer & Sandholm, 2011). Let o be a 
regular ESS for an evolutionary game G. Then, o is 
asymptotically stable under 


* any impartial pairwise comparison dynamic for G; 
* any separable excess payoff dynamic for G; 
* the best response dynamic for G. 


While the generality of this result is noteworthy, it must 
be stressed that what this establishes is a validation of “the 
use of regular ESS as a blanket sufficient condition for local 
stability under evolutionary game dynamics” (Sandholm, 
2010b). Local stability is not necessarily a good guide to 
the expected behavior of a population in an evolution- 
ary game. Consider, for example, the game shown in fig- 
ure 9.3.9: although S; is a regular ESS and is asymptotically 
stable, the basin of attraction for $;—that is, the set of all 
points that converge to S; as t > »—can be made arbitrarily 
small by adjusting the value of ¢ accordingly. 

Furthermore, it can also be the case that the basin of 
attraction for a population state is the entire interior of 
the simplex space, even though the population state is 
not asymptotically stable. Figure 9.3.10 illustrates a game 
where, under the replicator dynamics, the long-term 
behavior is such that the population will always con- 
verge to A. Yet A is not asymptotically stable, as small 


Figure 9.3.8 

For the game of Rock—Paper-Scissors, the population state 
in which all three strategies are equally represented is a 
neutrally stable state of the replicator dynamics. 


Evolutionary Game Theory 


Si Sy Ss 
Si (0, 0) (1, 0) (€, 0) 
S2 (0, 1) (1, 1) (-0, 0) 
S3 (0, €) (0, 0) (2€, 2e) 
(a) The payoff matrix. 
Figure 9.3.9 


An ESS can have an arbitrarily small basin of attraction. 


Figure 9.3.10 


(a) The payoff matrix. 


(b) The basin of attraction for $; appears on the right 
of the simplex and can be made arbitrarily small by 
adjusting the value of ¢ accordingly. Nevertheless, 
S3 is a regular ESS and is asymptotically stable. 


0.21 


(b) The basin of attraction for S, is the whole interior of the 
simplex, even though S;, is not asymptotically stable. Small 
displacements from S; result in the population following a 
lengthy trajectory, which carries it very far away from the 
all-S, state before eventually returning. (The trajectories 
ultimately converge to the all-S, point; the trajectories 
appear to stop in the middle of the simplex due to 

numeric limits on the differential equation solver.) 


A population state can be the expected evolutionary outcome without being asymptotically stable. 
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displacements may result in the population being carried 
very far away before eventually returning via a circuitous 
route. 

Evolutionary game theory provides a great variety of 
tools for modeling the boundedly rational behavior of 
populations in strategic contexts. Yet, as we have seen, 
the evolutionary outcomes may not always be rational: 
strictly dominated strategies may be played with posi- 
tive probability, as figure 9.3.7b shows. The fact that a 
population state is a (regular) evolutionarily stable state 
does not guarantee that it is, in any way, the expected or 
even likely evolutionary outcome, as seen in figure 9.3.9. 
And the long-term convergence behavior of a population 
can also be decoupled from asymptotic stability, as in fig- 
ure 9.3.10. With great variety comes great complexity and 
hence the need to exercise care when drawing inferences 
about expected outcomes. 
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Notes 


1. It should be noted that, in his collection of essays On Evolu- 
tion, published in 1972, Maynard Smith gives the same defini- 
tion of an ESS in the essay “Game Theory and the Evolution 
of Fighting.” But he clearly notes in the introduction to the 
volume that “I would probably not have had the idea for this 
essay if I had not seen an unpublished manuscript on the evo- 
lution of fighting by Dr George Price. ... Unfortunately, Dr 
Price is better at having ideas than at publishing them. The best 
I can do therefore is to acknowledge that if there is anything in 
the idea, the credit should go to Dr Price and not to me.” 


2. Whether this would, in fact, happen depends on specific 
features of the dynamics. 


3. In their 1973 paper, the game is called the “Hawk—Mouse” 
game as a result of George Price’s objection to using the term 
“Dove,” due to its religious connotations. 


4. It is assumed that the losing Hawk is selected at random 
with equal probability, which is why the payoffs in the upper- 
left cell of the payoff matrix take the form that they do. 


5. The two additional rest points are where the entire Row 
population follows S,; and the Column population follow S$, 
and vice versa. These are unstable rest points, but they exist 
because the replicator dynamics cannot introduce strategies 
into the population if they are originally absent. 
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9.4 Rationality in Economics: Theory and Evidence 


Sanjit Dhami and Ali al-Nowaihi 


Summary 


We examine the various senses in which economists use 
the term “rationality” in various domains such as cer- 
tainty, risk and uncertainty, time-discounting, and stra- 
tegic interaction. Although the rationality definitions 
in economics are precise and axiomatic, there has been 
much misunderstanding about them. For this reason, we 
also outline some of the commonly drawn implications 
and auxiliary assumptions and separate these from the 
axiomatic definitions. Finally, we confront the rational- 
ity axioms and the implications with the empirical evi- 
dence, drawing on insights from the exciting new field 
of behavioral economics. 


1. Outline of the Chapter 


Rationality has diverse meanings in the social sciences 
and in common discourse. The dominant paradigm in 
economics, neoclassical economics, defines rationality in 
a precise way. This forms the basis of a coherent body 
of economic theory that allows for sharp and testable 
predictions. However, economists typically justify their 
assumption of rationality on grounds of plausibility and 
tractability rather than on the only admissible grounds 
in science: conformity with the empirical evidence. 

The plan of this essay is as follows. Section 2 speci- 
fies the meaning of rationality in economics. Section 3 
considers some auxiliary assumptions that are either 
regarded as part of a broader concept of rationality or are 
sometimes mistakenly believed to be consequences of 
rationality as defined in section 2. Finally, section 4 pits 
the neoclassical view of rationality against the evidence 
that has emerged over the past several decades from the 
exciting and fastest-growing field in economics: behav- 
ioral economics. It is impossible to cite all the papers 
in such a short essay, so the reader is alerted to suitable 
surveys that contain other relevant references. 


2. Rationality in Economics 


This section explains the precise nature of rationality in 
economics. In each case, the primitives are the prefer- 
ences that individuals have over appropriately defined 
sets of objects. 


2.1 The Assumption of Rationality under Certainty 
Suppose that an individual has a set X CR” of choices, 
where x GX may be interpreted as a bundle of n goods 
or services. Let > be a binary preference relation: for 
x, y€X, x=y means that x is considered “at least as good 
as” y. 


Definition 1. The individual is rational, or equiva- 
lently, the preference relation > is rational, if the fol- 
lowing hold: 


For all x,y€X, we have either x=y or y=x. 
(Completeness) 


Let x,y,zEX with x=y and y=z. Then x=z. 


(Transitivity) 


Definition 2. If x=y, then we say that x is at least as 
good as y. Ifx=y and y =x, then we say that x is indif- 
ferent to y, and we write this as x~y. On the other 
hand, if x>y but it is not the case that y> x, then we 
say that x is strictly preferred to y and we write x > y. 


The assumption of rationality states that a decision maker 
is rational in the sense of definition 1, that is, a decision 
maker has a preference relation > on the set X of choices 
such that pairwise comparisons between any two alter- 
natives can be made (completeness) and preferences are 
transitive. 

Note that a violation of rationality (definition 1) in 
itself does not entail a violation of any law of logic. 
There is a widespread consensus among economists 
that the assumption of rationality is plausible. However, 
the crucial question is whether it conforms to human 
behavior. We examine the evidence in section 4. 
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Also, note that the definition of rationality, definition 
1, can be stated for any nonempty set X, not just X CR”. 
For example, X could be a set of lotteries (see subsection 
2.2, below) or consumption streams (see subsection 2.3, 
below). However, restricting X to be a subset of IR” allows 
us to introduce a useful continuity property (without 
explicitly defining a topology on X). 


Definition 3 (Continuity). The preference relation > is 
continuous if, for each y EX, both the sets {x EX: y = x} 
and {z €X: z> y} are closed. 


A utility function is simply a useful, and equivalent, 
alternative way of expressing the preference relation >: 


Definition 4 (Utility Function). A utility function 
u: X — R represents the preference relation >, if for 
x,y EX, x>=y if, and only if, u(x) = u(y). 


But does such a utility function exist? 


Proposition 1. Suppose that the preference relation > on 
X is complete, transitive, and continuous. Then there 
exists a continuous utility function u that represents > . 


The converse of proposition 1 also holds: 


Proposition 2. If a continuous function u represents 
the preference relation > on X, then > must be com- 
plete, transitive, and continuous. 


It is often more convenient to work with a utility func- 
tion than with preferences. 


2.2 Rationality under Risk 

Risk is pervasive in most economic problems. Let X = {x1, 

Xo,..., Xy} be a fixed finite set of real-valued outcomes 

such that x} <x2.<.. 
A simple lottery, or simple gamble, L, is a shorthand way 

of capturing risk: 


i 
2 SXp. 


3 Xny Pn), (1) 


where pi,..., Pn are the respective probabilities corre- 
sponding to the outcomes x1,..., X, such that p; €[0, 1] 
and Ya p; =1. The lottery (xj, 1) denotes an outcome x; 
received with certainty; this is the situation considered 
in subsection 2.1. Basically, a compound lottery is a lottery 
whose outcomes are themselves lotteries rather than real 
numbers. 

Let & be the set of all lotteries over X (simple and com- 
pound). Which lottery will the decision maker choose? 

The most widely used theory under risk in eco- 
nomics is expected utility theory (EU). Suppose that the 
decision maker is endowed with a binary preference 
relation over £ denoted by =. So, for Li, L2 EL, L2= Li 


L = (X1, Pi; X2, pr; - « 
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means that the lottery Lz is “at least as good as” the 
lottery L. 
Consider the following axioms on behavior under risk: 


Axiom 1 (Order). Order requires the following two 
conditions: 


For all lotteries Li, Lz, either L2>Li or Li>Lx. 
(Completeness) 


For all lotteries Li, Lz, L3: if L3=>L2 and L2>1;, then 
L3=L,. (Transitivity) 


Two further binary relations between lotteries can be 


defined in terms of >: 
Ii~I2@122=L, and L,>=L; (Indifference) 


L2>L; Sit is not the case that L; = Lo. 
preference) 


(Strict 


Axiom 2 (Best and Worst). x; > X1 (i.e., (%n, 1) > (%41, 1)). 


Axiom 3 (Continuity). For each lottery L, there is a 
p €[0, 1] such that L ~ (x1, 1 - p; Xn, p). 


Axiom 4 (Independence). For all lotteries L;, Lz, L, and 
all p €[0, 1], Lz = Li (L2, p; L, 1 - p) = (Li, p; L,1-p). 


The axioms of completeness, transitivity, and continu- 
ity introduced above are analogous to those introduced in 
subsection 2.1. From axiom 2, under certainty, a decision 
maker strictly prefers the highest to the lowest outcome. 
The continuity property in axiom 3 asserts that every lottery 
(no matter how complex) is equivalent to a simple lottery 
obtained by probabilistically mixing the highest outcome, 
X,, and the lowest outcome, x;. The independence axiom, 
axiom 4, requires that if a lottery L2 is preferred to a lottery 
Ly, then mixing each with a third lottery L (with the same 
mixing probability p) does not alter the preference. 

The next definition gives “rationality” a precise mean- 
ing under risk: 


Definition 5 (Axioms of Rationality under Risk). The 
axioms order, best and worst, continuity, independence, 
and reduction are collectively termed the axioms of 
rationality under risk.’ 


Proposition 3 (von Neumann—Morgenstern Expected 
Utility Representation). Suppose that the binary rela- 
tion > on the set & of lotteries satisfies the axioms 
of rationality under risk (definition 5). Then there is 
a von Neumann-Morgenstern utility function EU: 
LF — R that represents >, that is, L2>L, if, and only 
if, EU(L2) = EU(Li). This function takes the form 


EU(L)= ¥ pur), (2) 


i=1 
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where u(x;) is a real number assigned to the outcome 
X;, also called the utility of xj. 


The converse of proposition 3 also holds. This frame- 
work can be extended to uncertainty (where probabili- 
ties are subjective, rather than objective; Dhami, 2016, 
section 1.3; Savage, 1954). 

Finally, it is assumed that all probabilities satisfy the 
Kolmogorov (1950) axioms of probability. 


2.3 Rationality in Choices Made over Time 

Let ZCR be the set of possible outcomes and let TCR, be 
the set of corresponding times at which these outcomes 
are realized. Elements of the set Z are denoted by z; (j = 1, 
2,...) and elements of the set [ by & (j=1, 2,...). 
Denote by (z;, tj) an outcome-time pair, which guarantees 
the outcome z; at time ¢. Let = denote the binary prefer- 
ence ordering “at least as good as” on the Cartesian prod- 
uct X = ZxI. Notice that we have now redefined the set 
X from subsection 2.1 as outcome-time pairs. Also, let > 
and ~ denote, respectively, the binary relations “strictly 
preferred to” and “indifferent to” on the set X = ZxI. The 
following five axioms B1-B5 from Fishburn and Rubin- 
stein (1982) are typically called the axioms of rationality 
for time-discounting: 


Definition 6 (Axioms of Rationality for Time- 
Discounting). Let Zn, Z,€Z and ty, t; EY. 


Bl. Ordering: = is complete and transitive. 


B2. Monotonicity: If Zm> Zn, then (Zm,t)>(Zn,t) for 
any tI (larger rewards are preferred, keeping 
time fixed). 


B3. Impatience: Let t, < t, and z €Z with z > 0. Then, 
(z,t,) > (Z, tp) (the same outcome is preferred at 
an earlier date). Also (0, t,) ~ (0, tp) (no time pref- 
erence for zero rewards). 


B4. Continuity: Fix some outcome-time pair (zj, ti) 
ZxI. Then {(z,0): (z,0 = (z,f)} and {(z,0: (ZH) = 
(z, t)} are closed. 


BS. Stationarity: If (Zm, tq) ~ (Zn tp), then (Zm, t+ 0 ~ 
(Z,, tp + 0) for any tET (indifference between two 
outcomes depends only on the length of the 
time difference between the two). 


We have already encountered the analogues of axioms 
B1 and B4 in subsection 2.1. Axioms B2 and B3 are emi- 
nently reasonable. Axiom BS is crucial to producing the 
additive separability under exponential discounting. 
Fishburn and Rubinstein (1982) write, “However, we 
know of no persuasive argument for stationarity as a 
psychologically viable assumption” (p. 681). 
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The utility representation of the axioms of rationality 
for time-discounting is given next: 


Proposition 4 (Fishburn & Rubinstein, 1982, p. 682). 
If B1-B5 hold, then for any 6€(0, 1), there exists a 
continuous increasing real-valued function u defined 
on the domain Z such that 


(i) For all (2m,tq), (Zn, tp) in ZxT, (Zn, tq) = (Zn, tp) if, and 
only if, 


S'1U(Z,) =O? u(Z,,). 


(ii) The outcome-time pair (0, f) is treated as 5'u(0) = 0. 
(iii) Given 6, u is unique up to multiplication by a posi- 
tive constant. 


The axioms of rationality for time-discounting are testable; 
the evidence is considered in section 4. 

In the light of proposition 4, the resulting model is 
called the exponentially discounted utility model (EDU). 


2.4 Rationality in Game Theory 

Game theory deals with strategic situations in which indi- 
viduals mutually influence the payoffs of others through 
their actions and beliefs. Rationality plays a key role in 
the development of the central concepts in game theory. 
These are already developed in chapter 9.1 by Albert and 
Kliemt, chapter 9.2 by Perea, and chapter 9.3 by Alexan- 
der (all in this handbook), so this discussion is omitted. 
The chapter assumes a knowledge of the basic equilib- 
rium concepts (see, e.g., Fudenberg & Tirole, 1991). 

In the simplest setting of a static game of complete infor- 
mation, a Nash equilibrium places no restrictions on the 
beliefs of players. Such belief restrictions are dealt with in 
the epistemic foundations of a Nash equilibrium, and they 
turn out to be extremely demanding and unlikely to be 
met in practice. Suppose that the following conditions 
hold: mutual knowledge of the payoffs, mutual knowl- 
edge of rationality of players, common knowledge of the 
beliefs (or conjectures) each player has about the others, 
and common priors. Then, Aumann and Brandenburger 
(1995) showed, the common conjectures about what 
other players will do are in agreement and constitute 
a Nash equilibrium of the game. Polak (1999) showed 
that if mutual knowledge of the payoffs is strengthened 
to common knowledge of payoffs, then the Aumann- 
Brandenburger conditions imply common knowledge of 
rationality. Recent research offers an unsettled view of 
the epistemic foundations (Gintis, 2009). 

If the predictions of a Nash equilibrium and its refine- 
ments are violated by the empirical evidence, then so 
are the epistemic conditions. We briefly summarize the 
empirical evidence in section 4. 
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A Nash equilibrium is often justified as the outcome of 
a learning process. However, such processes do not guar- 
antee convergence of play to a Nash equilibrium, and 
when convergence does take place, there is no guarantee 
that the underlying learning process is plausible (Fuden- 
berg & Tirole, 1991, pp. 25-29). Even when convergence 
does take place under a plausible learning process, there 
is no guarantee that it does so in a reasonable amount of 
time that is representative of real-world interactions. 


3. Some Auxiliary Assumptions 


In this section, we briefly outline some auxiliary assump- 
tions that are commonly used in economics in addition 
to the assumption of rationality, given in section 2. 
These assumptions are sometimes mistakenly believed 
to be consequences of the rationality assumptions of sec- 
tion 2. At other times, they are regarded as part of a more 
restricted concept of rationality. 


3.1. Individuals Have Self-Regarding Preferences 
Individual i is said to have other-regarding preferences 
(or social preferences) if the individual cares about own- 
consumption, x;, and the consumption of others, x, that 
is, individual i’s utility function is u,(x;, x). In definitions 1 
and 4 and propositions 1 and 2, preferences are purely self- 
regarding since the utility of any individual i only depends 
on own-consumption; that is, instead of uj(x;, x1), we could 
as well just talk about u(x;). This has led to a common nart- 
rative that rationality implies self-regarding preferences. 

This, however, is not correct. One may define the ini- 
tial set X in any way one wishes to. For instance, for indi- 
vidual i, elements x ©X may be appropriately defined as 
pairs of the form (xj, x_;). One can then proceed, as in sec- 
tion 2, to derive a utility function with other-regarding 
preferences. However, in actual practice, neoclassical 
economics has not taken this direction.* 

All this has changed with the advent of behavioral 
economics, in which other-regarding preferences play a 
central role. We now know a great deal about the nature 
of other-regarding preferences through lab, field, and 
neuroeconomic evidence. Behavioral models of other- 
regarding preferences have made several precise and 
confirmed predictions. Experiments have played a key 
role in the precise identification of other-regarding pref- 
erences. Experimental games such as the dictator game, 
the ultimatum game, the public goods game, the trust 
game, and the gift-exchange game, among many others, 
have allowed stringent tests of competing theories. For 
surveys, the interested reader may consult Fehr and 
Schmidt (2006) and Dhami (2016, part 2). 
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3.2 There Is No Role for Emotions in Decision Making 
Rationality in economics has also been taken to imply 
cold, emotionless deliberation. This does appear to be an 
indirect implication of definitions 1, 5, and 6. None of 
the axioms explicitly takes account of emotional states. 
In contrast, there has been growing interest in modeling 
emotions in behavioral economics. 

In principle, one could modify the classical framework 
by introducing a set M of emotional states and, for any 
me€M, index the preference relation 2 in section 2 by m 
to get Z,; this applies to certainty, to uncertainty, and to 
the time dimension. This allows one’s preferences to be 
dependent on emotions. In an emotional state, one may 
make erratic or inconsistent decisions. In anger, one could 
signal credibly one’s own future intended actions. Anxiety 
is an important emotion that has been used to explain 
the equity-premium puzzle in finance (Caplin & Leahy, 
2001). Projection bias, the tendency to project our current 
emotional states to future states of the world, explains 
why those awaiting kidney transplants have relatively 
lower reported well-being as compared to a healthy con- 
trol group, yet after the transplant, both groups report 
similar well-being (Loewenstein, O’Donoghue, & Rabin, 
2003). For a large number of other applications, and for 
the relevant evidence, the interested reader can consult 
Dhami (2016, part 6). Behavioral game theory has used 
emotions to successfully explain diverse economic phe- 
nomena under the rubric of psychological game theory 
(Dhami, 2016, section 13.5). 


3.3. Perfect Attention, Unlimited Computing Power, 
and No Misperceptions 
The choice set X, whether it be the space of outcomes, lot- 
teries, or outcome-time pairs, can be infinite (section 2). 
Thus, the rationality assumption requires individuals to 
make a large number of pairwise comparisons in order 
to choose the most preferred or maximal element in the 
set X. In addition, transitivity must not be violated. This 
may require enormous computation power, memory, and 
attention. In contrast, the evidence shows that people are 
subject to inattention, misperception, and limited com- 
putation powers (see section 4 below). 

Consider the following typical example of what 
underlies almost all areas of economics: 


Example 1. A self-regarding individual lives for T time 
periods and cares only about the dated stream of his 
consumption, that is, c1, co,..., cr. At time t, t= 1, 
2,..., 7, he inherits a stock of savings, S;-1, from the 
previous period. This gives him the interest payment 
rS:1, where r= 0 is the return on one-period bonds. In 
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addition, he has income y; from other sources (e.g., 
labor income). Thus, his total period-t income is 
¥r+1S.1. This is spent on consumption c, and the bal- 
ance, S:= Vit 1Si1-—C, is saved. Thus, S; = S-i+ 5; Let 
his utility from period-t consumption be u(c,) and his 
subjective discount factor be 6 €(0, 1). His objective at 
time tis to choose (, Cu1,..., Cr SO aS to maximize the 
present value of his stream of utilities s d'u(c;) . 
At the time of his death, in period T, he cannot be in 
debt, so we impose the condition S; = 0. 


The problem formulated in example 1 can be given the fol- 
lowing mathematical form that we refer to as “problem-T”: 


For each time t= 1, 2,..., T, choose cy, Cui, ..., Cr SO 
as to maximize 

i=T . , re . 
y,., 6 'u(c,) (objective function) 


subject to the following conditions: 


Initial condition: 
Si given. 


Dynamic equation: 


Si=yit (14 nSi4- Ci, fori=t,t+1,..., T. 


Terminal condition: S;=0. 


Problem-T is a simple example of a dynamic optimization 
problem. It is an optimization problem because we have an 
objective function we want to maximize. It is a dynamic 
problem because excessive consumption (say) in the early 
periods will not leave enough money for adequate con- 
sumption in the final periods. The initial state of the con- 
sumer is his initial stock of savings, So. His final state is 
zero savings (because it is suboptimal to leave unused sav- 
ings). The control variables are the consumption levels. 

The solution method for problem-T is by backward 
induction. The consumer should start with the final period 
T. Then for every possible value in the stock of savings, 
Sri, inherited from the penultimate period, T-1, find 
the optimal level of consumption by choosing cr so as 
to maximize u(cr) subject to yr+ (1 +nSri-cr = 0. Then 
move back in time and repeat for period T- 1. The con- 
sumer should continue until reaching the first period, 
t = 1, and then substitute for the initial value for savings, 
So. This will then determine the optimal consumption 
levels cy,c3,...,Cz. There are, however, three problems 
that we now turn to: 


1. Although backward induction is conceptually simple, 
it requires a huge number of calculations. For exam- 
ple, consider a 40-year horizon and a period length 
of one year (so T = 40). Suppose that the income j; at 
each possible time t¢ can take two possible values, high 
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and low, and corresponding to this, we have just two 
possible optimal levels of consumption, low and high. 
Depending on the values of consumption, income 
and initial wealth, this could require the consumer to 
carry out up to 2*° > 10" (one trillion) calculations. 

2. Even when the problem is simplified, say, for a three- 
year horizon, so that the total number of calculations 
is, at most, 2° = 8, experimental subjects do not use 
backward induction (see Dhami, 2016, part 4). 


3. Any real-life situation is far more complex than exam- 
ple 1. For example, individuals might be uncertain of 
the time of death, future incomes, and future rates 
of return. Bequests might have to be calculated opti- 
mally. Decisions might be taken in hot, and unantici- 
pated, emotional states, and capital markets might be 
imperfect. 


The maintained assumption in neoclassical econom- 
ics is that the man on the street can solve these hard 
problems literally, in his head, in an instant (analytically 
or numerically) or behave in a manner as if he can solve 
the problem. However, when tested, the assumptions 
and predictions of many leading neoclassical models fail 
(see section 4). 

Furthermore, in many real-life situations, one might 
be distracted by other pressing issues (limited time, 
information overload, deadlines, marital discord, emo- 
tional lows) that make the supposed optimization exer- 
cise even more challenging. Yet, in real life, individuals 
routinely make complex decisions in the presence of 
uncertainty. So how do they do it? Could it be that they 
do not use the sort of optimization that economists 
believe they engage in and use instead simple rules of 
thumb and various boundedly rational methods? See 
subsection 4.5, below, and Dhami (2016, part 7). 


3.4 We Can Ignore Irrationality in Financial 
and Other Markets 
In economics, financial markets are typically held out to 
be the epitome of efficiency. The dominant tradition in 
finance argues that we can ignore any irrationality (in 
the sense considered in section 2) in financial markets. 
This argument is based on two assumptions but with 
little empirical support (Dhami, 2016, chapter 21). First, 
although irrational individuals, if there are any, may 
make errors, these errors will cancel out in the aggregate. 
Second, irrational market participants (or noise traders), 
if there are any, will be driven out of the market by the 
actions of rational market participants (or arbitrageurs). 
The assumption of Bayesian rationality in finance is 
central to the efficient markets hypothesis (EMH), which 
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is the cornerstone of modern finance.* The EMH asserts 
that the price of an asset at any point in time equals the 
present discounted value of the stream of future incomes 
from that asset (i.e., its fundamental value). The evi- 
dence, however, is that humans, even professionals, are 
poor at Bayesian updating, and many assets can deviate 
for long stretches of time from their fundamental values 
(Dhami, 2016, sections 19.7 and 19.18 and chapter 21). 


4. The Evidence on Rationality in Economics 


In this section, we briefly consider the evidence on the 
rationality assumptions of section 2 and the auxiliary 
assumptions of section 3. 


4.1 Evidence on the Basic Assumptions of Rationality 
The most basic definition of rationality (definition 1) 
requires that preferences be complete and transitive. 
Both conditions are violated in experiments. Iyengar 
and Lepper (2000) exposed subjects to either a limited- 
choice condition (6 varieties of jams) or an intensive- 
choice condition (24 varieties of jams). There was no 
difference in the percentage of jams sampled in both 
conditions. However, nearly 30% of the consumers in 
the limited-choice condition purchased a jam jar. The 
corresponding number in the intensive-choice condi- 
tion was only 3%. This suggests that individuals may 
find it more difficult to make choices when there are too 
many options; this may violate completeness. 

The most famous of the violations of transitivity 
occurs in the experiments on preference reversals; this is a 
problem in the domain of risk. Consider two probabili- 
ties p < P, two outcomes z < Z, and the following pair of 
lotteries: 


P-bet = (0,1-P;z,P),  $-bet = (0,1-p;Z,p). 


The P-bet has a higher probability, P, of winning, and 
the $-bet has a higher prize, Z. The decision maker is 
given the following two tasks: 


1. Direct choice between the P-bet and the $-bet. 


2. Assign certainty equivalents to the P-bet and the $-bet; 
these are denoted, respectively, by Cp and Cs.° 


The typical empirical finding, much replicated, is that 
the P-bet is chosen over the $-bet in the first task and 
Cp < Cs in the second task (Lichtenstein & Slovic, 2006). 
Transitivity fails because $-bet ~ Cs > Cp~ P-bet > $-bet 
(where > denotes strict preference). 

Unlimited attention is implicit in the assumption of 
rationality (definition 1). However, there is growing inter- 
est in the implications of limited attention, which reflects 
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limited cognitive abilities of humans. Here are some find- 
ings from this literature (Dhami, 2016, section 19.17): 
consumers pay less attention to taxes that are not listed 
on price stickers but added at cash registers; displaying 
information about taxes on price stickers reduces sales. 
Individuals pay relatively less attention to electronic toll 
payments, allowing operators to levy higher tolls as com- 
pared to, say, cash paid at manual toll collections, which 
make the payment salient. Car sales and car prices are sen- 
sitive to integer multiples of the 10,000-mile threshold, so 
there is too large a gap in prices of cars with mileage read- 
ings of 9,999 and 10,000 miles because consumers pay too 
much attention to the leftmost digit. Limited attention is 
also increasingly used to explain suboptimal economic 
decisions of the poor, whose attention is diverted by the 
basic problems of food, shelter, and clean water, a form 
of cognitive taxes that reduce mental bandwidth (Datta & 
Mullainathan, 2014; World Development Report, 2015). 
There are now promising theoretical attempts to formal- 
ize limited attention in which consumers pay differential 
attention to various attributes of a good (Gabaix, 2014). 

Despite this evidence, the common consensus in eco- 
nomics, and in behavioral economics, is to maintain a 
minimum degree of rationality as dictated by definition 
1. A compromise could be to retain rationality in the 
sense of definition 1, but only within a suitably narrow 
frame or context (Gintis, 2009, 2017). However, it is not 
clear if all evidence that contradicts complete and transi- 
tive preferences could be explained by invoking differ- 
ent contexts and frames. 


4.2 Evidence on Rationality in Risk and Uncertainty 
The leading example of a violation of the independence 
axiom is the Allais paradox (Allais, 1953; Dhami, 2016, 
section 1.5; see also chapter 8.2 by Peterson, this hand- 
book). It is now well recognized that nonlinear probability 
weighting (NLPW) explains the Allais paradox. On the 
other hand, EU requires that probabilities enter into the 
utility function linearly. 

The evidence for NLPW has grown enormously over 
the years (Fehr-Duda & Epper, 2012). NLPW plays a cen- 
tral role in the modern alternatives to EU, such as rank- 
dependent utility (RDU) due to Quiggin (1982) and prospect 
theory (PT) due to Kahneman and Tversky (1979) and 
Tversky and Kahneman (1992). PT has been enormously 
successful in explaining the evidence, particularly where 
both EU and RDU fail. PT has been successfully incorpo- 
rated into formal economic models in many areas of eco- 
nomics; for surveys, see Kahneman and Tversky (2000) 
and Dhami (2016, part 1); see also chapter 8.3 by Gléck- 
ner (this handbook). 
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These are by no means the only refutations of EU.° 
Other refutations include the following (Dhami, 2016, 
chapter 3): unreasonable attitudes toward risk, the endow- 
ment effect, anomalies of tax evasion, the efficacy of goal- 
setting behavior, backward-bending labor supply curve of 
taxi drivers, the equity-premium puzzle, contract choice 
and renegotiation, and the pricing of assets and the skew- 
ness of asset returns. Furthermore, close genetic relatives 
of humans (e.g., primates such as capuchin monkeys) 
exhibit similar violations of EU, and their behavior sup- 
ports PT. The implication is that preferences supporting 
PT were hardwired into the common ancestors of humans 
and capuchin monkeys. 


4.3 Evidence on Rationality in Time-Discounting 
The utility representation of rational time preferences 
takes the form of the EDU (proposition 4, recall subsection 
2.3). Under EDU, the discount factor 6 is constant. Under 
EDU, the same discount factor 6 applies independently of 
(1) the magnitude of an outcome (absence of magnitude 
effect) and (2) the sign, positive or negative, of an outcome 
(absence of gain-loss asymmetry). Furthermore, preferences 
are independent of the shapes of consumption profiles. 
Strong empirical evidence refutes each of these fea- 
tures of the EDU model. Behavioral models of time- 
discounting have been proposed that are able to account 
for all of these violations. Consider the following viola- 
tions of the EDU model reported in Thaler (1981), which 
have been often replicated (see Dhami, 2016, chapter 9, 
for the updated evidence): 


1. Magnitude effect: Subjects reported being indifferent to 
$15 received now and $60 in a year’s time, $250 received 
now and $350 received in a year’s time, and $3,000 
received now and $4,000 in a year’s time. The implied 
respective annual discount factors are 0.25, 0.71, and 
0.75; these are not identical with those required by 
EDU. Thus, larger amounts are discounted less. 

2. Gain-loss asymmetry: The discount factors for losses 
turned out to be larger than those for gains, so losses 
are discounted less than gains. This could be because 
losses are more salient than gains. 

3. Common difference effect: When asked how much 
money in one month, one year, and 10 years, respec- 
tively, would make them indifferent to receiving $15 
now, the median response of subjects was $20, $50, 
and $100, respectively. The respective implied annual 
discount factors are 3%, 30%, and 83%. Hence, the 
shorter the horizon, the lower the implied annual dis- 
count factor (greater impatience); in contrast, under 
EDU, the annual discount rate factor stays constant. 
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The common difference effect suggests a taste for 
immediate gratification. A major thrust of the behav- 
ioral economics literature is to explain this phenom- 
enon. A tractable and popular alternative to EDU is 
quasi-hyperbolic discounting (Laibson, 1997; Phelps & Pol- 
lak, 1968) in which time preferences are represented by’ 


T 
Gp song HU tp > Cue), Op <1. (3) 
t=1 
In (3), except for the presence of 6 €(0, 1), the utility 
function is identical to the EDU model. The factor 6 
shrinks the value of future lifetime utility relative to cur- 
rent utility, thereby creating an additional present-bias 
for current consumption, or a taste for immediate gratifi- 
cation. It also leads to a pattern of increasing discount 
rates as we approach the present that potentially explain 
the common difference effect. 

A taste for immediate gratification creates self-control 
problems. Furthermore, people have imperfect aware- 
ness about their future self-control problems; hence, 
they often take inadequate preventive measures. This 
also leads to the time inconsistency problem (i.e., choices 
made now about a future date ¢ need not be optimal 
when the individual reoptimizes at time ft). Classically, 
in economics, time-inconsistent choices were consid- 
ered irrational (and are impossible under EDU), but evi- 
dence suggests that they often occur. 

The quasi-hyperbolic framework resolves several 
problems that are unexplained by the axioms of ratio- 
nality for time-discounting (definition 6). Some of these 
are as follows (for details, see Dhami, 2016, chapter 11): 
Why does consumption track income so closely? Why 
do individuals undersave for retirement? Why is there 
a sharp drop in consumption at retirement? Why do 
individuals hold illiquid assets and credit card debt 
simultaneously? What causes addictions? Why do some 
smokers pay more to buy smaller packs of cigarettes or 
support sales taxes on cigarettes? Why do some people 
buy annual gym memberships when they could save 
money by paying on a pay-as-you-go basis? 


4.4 Evidence on Rationality in Strategic Interaction 

Classical game theory (CGT) makes precise and testable 
predictions (see chapter 9.1 by Albert & Kliemt, this hand- 
book). However, behavior in the early rounds of games 
and in games unfamiliar to the players is often inconsis- 
tent with the predictions. Learning and experience may 
or may not produce data consistent with the predictions 
(Camerer, 2003; Dhami, 2016, parts 4 and 5). People make 
many important decisions only a few times in their lives 
and experience limited learning (e.g., the choice of a 
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university degree, a marriage partner, a house, consumer 
durables, or a pension plan). Firms also make many impor- 
tant decisions rarely (capital restructuring, mergers, sunk 
costs in major machinery and equipment, and choice of 
a new product). 

Even in decisions that are taken frequently, the envi- 
ronment is ever changing and uncertain and hence 
always novel. Thus, arguably the choices made in the 
early rounds of an experiment are vital in understanding 
strategic human behavior, yet these choices do not typi- 
cally conform to the predictions of CGT. Such data are, 
in many cases, much better explained by behavioral mod- 
els of game theory such as level-k models, quantal response 
equilibrium, analogy-based equilibrium, cursed equilibrium, 
team reasoning, and evidential reasoning (for details, see 
Dhami, 2016, chapter 13). 

We summarize some of the evidence here. It is imprac- 
tical to cite all the references in such a short chapter; 
see the book-length treatments in Camerer (2003) and 
Dhami (2016, part 4) for the details. 


1. In games requiring more than two steps of iterated 
deletion of dominated strategies, a majority of the 
players violate the predictions of CGT. 


2. Extensive-form games and their equivalent normal- 
form representations often elicit different behavioral 
responses. 


3. The backward induction prediction of CGT in 
dynamic games of full information is refuted by the 
evidence, even when subjects highly trained in back- 
ward induction (e.g., chess grandmasters) play centi- 
pede games. 

4. Even when the simplest mechanisms are tested in 
experiments (e.g., the Abreu—Matsushima and the 
Glazer—Perry mechanisms, which are dominance- 
solvable), the results are unsupportive of CGT. This 
casts doubt on the implementability of more complex 
mechanisms. 


5. Subjects play mixed strategies in experiments but 
not in the proportions predicted by a mixed-strategy 
Nash equilibrium. There is some positive evidence 
from sports contexts, but there are many confound- 
ing factors in such experiments, and it is not clear 
if such motor skills have external validity for other 
economic contexts (e.g., choice of pension plans). 


6. In games with multiple equilibria, none of the 
selection principles in CGT, such as payoff domi- 
nance or risk dominance, fully account for the data. 
The equilibria are strongly history dependent. Preplay 
communication enhances coordination. 
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7. The CGT predictions of alternating offers bargain- 
ing games are not supported by the evidence. When 
players reject an offer, they often make counteroffers 
that give them an even lower share (disadvantageous 
counteroffers). The predictions of bargaining under 
one-sided asymmetric information in CGT are not 
supported, but there is surprising and unexplained 
support for the predictions of two-sided asymmetric 
information under CGT. Under unstructured bar- 
gaining, which is more realistic, factors unimportant 
in CGT, such as self-serving bias, explain bargaining 
impasse under full information. 

8. The pattern of searches and lookups in Rubinstein’s 
alternating offers bargaining game using the MOUSE- 
LAB software does not support the predictions of CGT. 


9. Insignaling games, there is no support for the predic- 
tions of CGT beyond the Cho-kKreps intuitive criterion. 


10. There is evidence that several emotions, such as 
guilt-aversion, surprise-seeking, anger, and inferring 
intentions behind the actions of others, directly 
enter into the utility functions of players. This runs 
counter to CGT and forms the subject matter of the 
promising approach in psychological game theory. 


4.5 Heuristics and Biases: An Alternative 

to Optimization 

As noted in subsections 2.2 and 3.3, the dominant view 
in economics is that people are fully rational Bayesian deci- 
sion makers. By this, it is meant that they use unbounded 
rationality, have unlimited attention and computing 
abilities, optimize using all appropriate mathemati- 
cal and statistical tools, and obey the laws of standard 
(Kolmogorov) probability theory (not just Bayes’ rule). 
Tversky and Kahneman (1971, 1974) put this view to the 
test, and this has spawned an enormous literature, called 
the heuristics and biases research program (see also chap- 
ter 2.4 by Fiedler, Prager, & McCaughey and chapter 8.5 
by Hertwig & Kozyreva, both in this handbook). 

The evidence showed that individuals are far from 
fully rational Bayesian decision makers, but their behav- 
ior is not random either. Individuals do not take account 
of all available information, do not correctly use Bayes’ 
law and other laws of statistics, and eschew mathematical 
optimization in favor of simple rules of thumb or heuris- 
tics. Most heuristics are fast, in terms of the computation 
time required, and frugal in the use of information. Since 
these heuristics do not optimize, their performance is 
usually suboptimal. Even statistically sophisticated 
researchers and experts typically rely on these judgment 
heuristics.* Gerd Gigerenzer and collaborators have shown 
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that when there is uncertainty about the optimal solu- 
tion, the performance of heuristics compares favorably 
with more complex mathematical or statistical methods 
(see Dhami, 2016, section 19.15). 

When individuals use the representativeness heuristic, 
they compare the likeness of even small samples with 
features of the population distribution. Hence, they 
assign too high a probability that small samples share 
the properties of large samples and behave “as if” they 
obeyed a law of small numbers, rather than the statisti- 
cally correct law of large numbers. Thus, when asked to 
produce random sequences, the constructed sequences 
show up too much negative autocorrelation. This creates 
the gambler’s fallacy (reluctance to bet on previously win- 
ning numbers). The converse, a belief in positive autocor- 
relation in outcomes when there is none, is known as 
the hot hands fallacy (basketball and football players who 
score heavily in a game are often perceived to be on a 
hot streak even when there is none). The implications 
for financial markets (and generally for economics) are 
profound, and this provides a much better explanation 
of behavior in financial markets (Dhami, 2016, part 7). 

Other heuristics that are increasingly used in eco- 
nomics and finance to explain human behavior include 
the availability heuristic, the anchoring heuristic, base rate 
neglect in using Bayes’ law, conservatism or underweighting 
of the likelihood of a sample, hindsight bias, confirmation 
bias, and regression to the mean.’ In particular, the failure 
of Bayes’ law is problematic for all areas of economic 
theory because Bayesian rationality lies at the heart of 
modern economic theory. 

It is often claimed that presenting information in a 
frequency format rather than in a probability format elimi- 
nates biases arising from using heuristics (Cosmides & 
Tooby, 1996; Gigerenzer & Hoffrage, 1995). While this 
is true for some heuristics, the case for the efficacy of 
the frequency format is vastly overstated (Dhami, 2016, 
chapter 19). Further, most real-world economic informa- 
tion is arguably presented in a probability format. 

Cognitive limitations force individuals to group dif- 
ferent events into separate categories and to treat all 
events in the same category in an identical manner. This 
insight plays a central role in new behavioral equilib- 
rium concepts, such as analogy-based equilibrium and 
cursed equilibrium, as well as for understanding persuasion 
and advertising (Dhami, 2016, sections 13.7 and 19.12). 

People form mental models of the world that can take 
the form of beliefs about causal relations, social identities, 
categorizing disparate information into coarse catego- 
ries, and social worldviews (see chapter 2.3 by Johnson- 
Laird, this handbook). Mental models are pervasive, help 
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economize on cognitive costs, are transmitted from gen- 
eration to generation, are typically inertial once formed, 
and can both assist and deter optimal decisions, depend- 
ing on their accuracy (Datta & Mullainathan, 2014; 
World Development Report, 2015). Humans are hard- 
wired with some mental models, while in other cases, 
culture plays an important role in their formation. This 
provides another alternative to optimization in making 
decisions (Dhami, 2016, section 19.3). 

Another alternative to Bayesian rationality is Her- 
bert Simon’s seminal approach to bounded rationality 
(Simon, 1979; see chapter 8.5 by Hertwig & Kozyreva, 
this handbook). In a direct reference to optimization, 
Simon (1979) famously notes, “But there are no direct 
observations that individuals or firms do actually equate 
marginal costs and revenues.” The Nobel Prize winner 
Reinhard Selten (2001) draws attention to the fact that 
economic problems are NP-complete, that is, the number 
of steps in the required solution grows exponentially with 
the size of the problem (e.g., the traveling salesman prob- 
lem).'° Even for apparently simple problems of this sort, 
the computing time required to solve the problem is very 
high. It then becomes a leap of faith that the man on the 
street can solve this problem or act “as if” he could. Yet, 
neoclassical economics does make that leap of faith. 

Herbert Simon proposes that instead of being Bayesian 
reasoners, individuals satisfice, that is, they have an aspira- 
tion level and adjust gradually in its direction through a 
sequence of steps (Simon, 1979, 2000; Selten, 2001). Empir- 
ical evidence is supportive of the theory (Caplin, Dean, & 
Martin, 2011): subjects appear to search sequentially and 
stop searching when their aspiration level is achieved. This 
can demonstrate how the cooperative outcome could be 
achieved as a solution to the prisoner’s dilemma game 
(Karandikar, Mookherjee, Debraj, & Vega-Redondo, 1998). 

Finally, the approach of Gerd Gigerenzer and col- 
leagues on fast and frugal heuristics is another worthwhile 
alternative to Bayesian rationality (see Dhami, 2016, sec- 
tions 19.14 and 19.15; Gigerenzer & Hoffrage, 1995). They 
show that heuristics can often outperform selected opti- 
mization methods, but in many cases, the optimization 
benchmark in these problems is far from clear. In some 
cases, these heuristics are derived from evolutionary adap- 
tation, but mostly they are provided by the experimenter. 


4.6 Evidence on the Efficiency of Financial Markets 

The efficient markets hypothesis, EMH (see section 3.4), 
is rejected by a very large body of empirical evidence. 
Each of the following empirical findings rejects EMH:'' 
prices of substitute assets differ consistently; there is 
gradual, rather than instantaneous, flow of information 
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in the stock market; asset prices exhibit momentum at 
the short horizon and mean reversion at the long hori- 
zon; asset bubbles, particularly in the housing market, 
are well documented; stock markets crash even when 
there is “no news”; and asset prices exhibit underreac- 
tion and overreaction to new news. Indeed, a mistaken 
belief in EMH can be disastrous for financial markets, 
as demonstrated by the decision of the chairman of 
the Federal Reserve, Alan Greenspan, to not intervene 
in financial markets before the crash of 2007, believing 
that markets are efficient. 

The refutation of both assumptions in the opening 
paragraph of subsection 3.4 and the discrediting of EMH 
on empirical grounds have led to the fast-growing disci- 
pline of behavioral finance. Behavioral finance recognizes 
that individuals are boundedly rational and follow a 
range of judgment heuristics that can give biased solu- 
tions (subsection 4.5). Contrary to the assertion in clas- 
sical finance, the errors of noise traders often aggregate 
in the same direction. 

In the event of a departure of prices from fundamental 
values and a strong belief on the part of noise traders that 
this mispricing will persist, rational investors will bet on 
the mispricing getting worse. Hence, the normal forces 
bringing prices back in line with fundamental values 
might not work, and we might observe bubbles on asset 
prices, or extended periods during which prices depart 
from fundamental values. The presence of institutional 
investors does not solve the problem either, because the 
deposits they receive from small investors depend on 
their past returns (performance-based arbitrage). Hence, 
they may liquidate their loss-making assets prematurely 
before the end of the year to demonstrate high returns, 
even when they ought to have held these assets longer. 

There is also a growing literature on corporate finance 
that applies the behavioral evidence on humans to CEOs 
of companies (Dhami, 2016, section 21.7). For instance, 
it has been demonstrated that overconfident CEOs may 
undertake mergers even when there is no economic 
case for doing so. This might well explain why so many 
mergers fail. 


5. Conclusions 


Neoclassical economics, which is still the mainstream, 
does well to propose precise and testable assumptions 
and rigorously derive their logical conclusions. However, 
the evidence (section 4) is not consistent with the basic 
assumptions of rationality (section 2) and the auxiliary 
assumptions (section 3). Building on empirical evidence 
and insights from cognitive psychology, behavioral 
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economics has made great strides in building a new eco- 
nomics that is as rigorous as neoclassical economics but 
in better conformity with the evidence. 
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Notes 


1. For ease of exposition, we restrict ourselves to a fixed finite 
set X of real-valued outcomes. Extensions to vector-valued out- 
comes are possible and to sets X that are not necessarily finite. 
See, for example, Kreps (1990), who also makes a strong case 
for exploring alternatives to expected utility theory. 


2. The reduction-of-compound-lotteries axiom is not formally 
stated above. For a formal definition, see Dhami (2016, p. 86). 


3. To be sure, there are some models of keeping up with the Jone- 
ses and snob consumption, but these have played only a fringe 
role in modern economics. 


4. Bayesian rationality augments the rationality assumptions 
from section 2 with the assumption that individuals update 
their beliefs when new information arrives, by using Bayes’ rule. 


5. The certainty equivalent is a guaranteed sum of money 
that makes the decision maker indifferent between accept- 
ing the money or the lottery. Thus, $-bet~ Cs and Cp~ P-bet, 
where ~ denotes indifference. 


6. The interested reader can peruse Kahneman and Tversky 
(2000), Starmer (2000), and Dhami (2016) for the details. 


7. See Dhami (2016, chapter 10) for alternative models that 
explain the common difference effect. 


8. For the claims in this paragraph, see Gilovich, Griffin, and 
Kahneman (2002); Kahneman (2003, 2011); Tetlock (2006); and 
Dhami (2016, part 7). 


9. For a discussion of these heuristics, evidence, incorporation 
into new theoretical models, and economic applications, see 
Dhami (2016, chapter 19). 


10. Suppose that a salesman is provided with a list of cities and 
the distances between each pair of cities. The objective, starting 
with a home city, is to find the shortest possible route such 
that the salesman visits each city exactly once and returns to 
the home city. 


11. For the claims made in this section and the relevant refer- 
ences, the interested reader can consult the surveys in Shleifer 
(2000) and Dhami (2016, chapter 21). 
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Section 10 Aspects of Social Rationality 


10.1 Social Epistemology 


Franz Dietrich and Kai Spiekermann 


Summary 


Social epistemology studies knowledge in social contexts. 
We distinguish two types, depending on the holder of 
knowledge. Epistemology in groups studies individuals 
learning from others, while epistemology of groups stud- 
ies groups as knowers, literally or metaphorically. Group 
knowledge can emerge explicitly, through aggregation 
procedures like voting, or implicitly, through institu- 
tions like deliberation or prediction markets. In the 
truth-tracking paradigm, group beliefs or actions aim at 
“correctness”—in virtue of facts that are empirical or nor- 
mative, real or constructed, objective or only intersub- 
jective, universal or relative, and so on. Procedures and 
institutions are evaluated by epistemic performance: Are 
they truth-conducive? Do groups become “wiser” than 
their members? This chapter reviews several procedures 
and institutions, discussing epistemic successes and fail- 
ures. Jury theorems provide formal arguments for epis- 
temic success, but only some have defensible premises. It 
will be argued that larger groups can outperform smaller 
groups, yet without becoming infallible. 


1. Scope and Problems of Social Epistemology 


Social epistemology is the branch of epistemology that 
studies knowledge in social contexts. This review first 
sets the stage by introducing clarifications, distinctions, 
and applications (section 1). It then discusses aggrega- 
tion procedures (section 2) and institutional arrange- 
ments (section 3) that generate collective outcomes that 
“track the truth” or lead to epistemic failures (section 4). 


1.1 Epistemology in Groups versus Epistemology 

of Groups 

A first distinction pertains to the knowledge holder, 
who is either a group member or the group as a whole. 
Epistemology in groups might be regarded as a species 
of individual epistemology, although its focus lies in 
how individual knowledge depends on social inputs; 


typical questions are rational responses to testimony 
and peer disagreement and “irrational” or subpersonal 
responses such as belief contagion through social media. 
Epistemology of groups ascribes knowledge to the group; 
let us talk here of social knowledge or belief (List & Pet- 
tit, 2011; see also Goldman & O’Connor, 2019; Schmitt, 
1999). The meaning and status of social knowledge or 
belief is controversial. The group in question can be more 
or less structured and cohesive, with implications for 
whether the group qualifies as an agent and for what it 
means for the group to believe or know something, rang- 
ing from literal to purely metaphorical meanings. Argu- 
ably, a committee, firm, or state (if treated as an agent) 
can believe something literally, while a prediction mar- 
ket or the random group participating in a survey can 
believe something only metaphorically. List (2014) dis- 
tinguishes between three types of social beliefs: aggregate 
beliefs are mere summaries of beliefs of group members; 
common beliefs are ultimately individual beliefs held by 
each group member, with a common awareness of these 
beliefs; and corporate beliefs are beliefs of the group in a 
literal sense, which presupposes that the group qualifies 
as an agent. Theoretical distinctions aside, it is evident 
that we routinely invoke social knowledge or beliefs, 
for instance, when saying that “we know” something 
or that “prediction markets knew” that Obama would 
become president of the United States. In a formal analy- 
sis, our distinction between epistemology in groups and 
of groups is sharp: the former is studied using belief revi- 
sion models, the latter is studied using aggregation mod- 
els. In this review, we focus on epistemology of groups, 
but for other aspects of social epistemology, see Gold- 
man and O’Connor (2019), Goldman and Whitcomb 
(2011), Goldman (1999), and Schmitt (1999).’ 


1.2 Narrow versus Broad Social Epistemology 

Narrowly construed, social epistemology addresses only 
social knowledge or belief: it addresses how a legal court 
learns whether the defendant is guilty, not how it decides 
whether to convict him; how a community of physicists 
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discovers a law, not how it decides whether to perform an 
experiment; and so on. Broadly construed, however, social 
epistemology addresses social decisions in a wide sense, 
covering not only beliefs but also, for instance, actions: 
courts passing sentences, scientists choosing experiments, 
parliaments passing laws, governments setting goals, com- 
missions working out ethical standards, and so forth. For 
such a social decision to be an epistemic matter, it must 
have the property of being correct or incorrect in virtue 
of some fact or truth. A court sentence may be an epis- 
temic matter, its being correct or incorrect depending on 
whether the crime has been committed and merits the 
sentence. Selecting Oscar winners might be an epistemic 
matter. One reason for adopting this broad notion of social 
epistemology could be that correct decisions often are, or 
can be rationalized as, the result of beliefs. This chapter 
uses “(social) decision” in the widest sense, to include, for 
instance, social beliefs and social actions.’ To be an object 
of social epistemology, broadly construed, the decision 
must, however, be correctness-apt.* 


1.3 The Truth to Be Tracked 

One can distinguish between three different types of 
facts that make social beliefs or social decisions more 
broadly correct or incorrect: logical, empirical, and nor- 
mative facts. For example: 


* The mathematical community forms mathematical 
beliefs. Correctness depends only on logical facts. 


* The monetary policy committee of a central bank pre- 
dicts whether inflation will rise. Correctness depends 
on empirical facts, besides logical facts. 


+ A group of doctors decides whether to shift funds from 
medical research to patient treatment. Correctness 
depends on normative and empirical facts, besides logi- 
cal facts. 


Social epistemology with normative facts is contro- 
versial, since the existence and the nature of normative 
facts are debatable.* But epistemology about normative 
facts should not be dismissed, because truth-tracking is 
meaningful regardless of whether normative facts are 
real or constructed (e.g., socially constructed), objective 
or at least intersubjective (within the group), universal 
or relativistic (as in cultural relativism), and natural or 
non-natural. The fact—normative or other—should, 
however, be suitably stable and procedure-independent. 
If the fact is of a constructed kind, then its existence and 
nature should not emerge from the social procedure but 
precede that procedure; otherwise, there is no truth to be 
tracked, but one to be constructed. 


Franz Dietrich and Kai Spiekermann 


1.4 Truth-Tracking and Rationality 

Rationality, understood here as internal coherence (i.e., 
formal rather than substantive rationality), is necessary 
but not sufficient for correctness. Beliefs can be perfectly 
consistent yet false, betterness judgments can be transi- 
tive yet misguided, and so on.° Rationality is thus only 
one step toward correctness. But this step is already very 
difficult to take when the holder of the beliefs (judg- 
ments, evaluations, and so on) is a group rather than an 
individual. Importantly, rationality of the group mem- 
bers does not guarantee rationality of the group, as we 
know from many aggregation paradoxes, such as Con- 
dorcet’s voting paradox and the discursive dilemma, and 
from general impossibility results such as Arrow’s Theo- 
rem (see section 2.1 and also chapter 10.2 by Schmid, this 
handbook). The sheer difficulty of achieving social ratio- 
nality might explain why much of social choice theory 
focuses on social rationality, setting aside correctness. This 
limited focus does not place social choice theory outside 
the social-epistemological endeavor, however, as rational- 
ity is necessary for correctness. The interaction between 
individual rationality, social rationality, and truth-tracking 
is complex. For example, irrational individual belief revi- 
sion can hinder individual truth-tracking while leading to 
a fruitful epistemic diversity that promotes social truth- 
tracking (see section 3.2). 


1.5 Social Knowledge, Justification, and Reliability 

Most well-known problems about individual knowledge 
carry over to social knowledge. For instance, it is contro- 
versial whether knowledge is justified true belief, a thesis 
threatened by “Gettier cases,” in which a subject (for us: 
a group) forms justified true beliefs by sheer coincidence 
(Gettier, 1963). It is also controversial whether justifica- 
tion to hold a belief comes from sufficient evidence or 
from a reliable procedure (see Goldman & Beddor, 2016, 
on evidentialism versus reliabilism). Despite their famil- 
iarity, such “knowledge problems” can take a distinctive 
form for social knowledge, notably because groups use 
different procedures than individuals to acquire beliefs 
(e.g., voting procedures). Nonetheless, this chapter shall 
set aside the question of what exactly social knowledge 
means and instead zoom in on how social beliefs and 
decisions more broadly can be formed and track the truth. 


1.6 Social Epistemology Applied 

to Epistemic Democracy 

Social epistemology is partly a positive theory of which 
institutions and procedures do track the truth. By con- 
trast, the core claim of epistemic justifications of democ- 
racy is normative: institutions and procedures should 
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track the truth to be legitimate (Estlund, 2008). By 
contrast, proceduralist justifications of democracy main- 
tain that institutions or procedures are not legitimate 
because they track any independent truth but because 
they are procedurally fair, which often means that voters 
should have an equal say and alternatives should get an 
equal chance. The difference is fundamental: if there is 
an Oracle of Delphi that always tells the truth, then the 
procedure of blindly implementing the oracle’s recom- 
mendation for society is epistemically good (it tracks the 
truth) but procedurally bad (it is totally undemocratic). 
The debate between epistemic and procedural theories 
of democracy is ongoing (for a hybrid position, see Peter, 
2007). For epistemic theories of democracy, social episte- 
mology is a highly relevant enterprise. 


2. Aggregation Procedures to Track the Truth 


This section turns to prominent procedures generating 
social beliefs (section 2.1) or other truth-tracking deci- 
sions (section 2.2) and then sketches how jury theorems 
help establish the truth-conduciveness of such proce- 
dures (section 2.3). 


2.1 Forming Social Beliefs or Judgments 
Aggregating beliefs on a single proposition A group of indi- 
viduals is interested in whether a given proposition p 
is true. For instance, an economic panel is interested in 
whether inflation will rise, or a jury in whether the defen- 
dant has committed the crime. Each group member holds 
a belief about p, in the binary form of “yes” or “no.” An 
aggregation procedure takes the individual beliefs about 
p as input and returns a social belief about p as output. 
Social abstentions are allowed: society can say “yes,” or 
say “no,” or abstain (but for simplicity, individuals cannot 
abstain). Here are some examples. Majority rule makes soci- 
ety believe what the majority believes (and abstain in case 
of a tie). Asymmetric supermajority rules make social belief 
in p harder to achieve than social disbelief by requiring a 
supermajority support for a social “yes” while otherwise 
opting for a social “no.” Symmetric supermajority rules make 
social “yes” and “no” equally hard to achieve by requir- 
ing the same supermajority support for each. This often 
results in social abstention, especially for high thresholds. 
Extreme cases of (asymmetric or symmetric) supermajor- 
ity rules are (asymmetric or symmetric) unanimity rules, in 
which the supermajority threshold is a unanimity thresh- 
old. Later, in section 2.3, we analyze how likely it is that 
resulting social beliefs track the truth. 

Aggregating beliefs about multiple propositions: Judgment 
aggregation Many groups are interested simultaneously in 
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the truth of different propositions, something referred to 
as a judgment-aggregation problem (e.g., Dietrich, 2007; List 
& Pettit, 2002). For instance, a court may need collective 
beliefs about three propositions: the defendant has com- 
mitted a certain act (p), such an act breaks the law (q), 
and the defendant has broken the law (r). A judgment- 
aggregation problem does not simply reduce to several 
belief-aggregation problems about a single proposition 
each, because the propositions in question are typically 
logically interconnected. If in our court example, the 
court comes to believe p and q, then it must believe r. 
Disbelieving r would be logically inconsistent; abstain- 
ing on r (neither believing nor disbelieving r) would be 
deductively unclosed. The trouble is that voting on each 
relevant proposition in isolation often generates incon- 
sistent and deductively unclosed social beliefs: even if 
each member of the court holds consistent and complete 
(and thus deductively closed) beliefs, it may happen that 
a majority believes p, another majority believes q, and yet 
a majority disbelieves r. This phenomenon is referred to 
as the doctrinal paradox (Kornhauser & Sager, 1986) or the 
discursive dilemma (Pettit, 2001). The discursive dilemma 
generalizes far beyond propositionwise majority voting. 
General impossibility theorems establish that, as soon 
as propositions are sufficiently interconnected, there exist 
no propositionwise judgment-aggregation procedures that 
are well behaved in certain senses; see in particular List 
and Pettit’s (2002) theorem and the Arrow-type theorem 
for judgment aggregation (Dietrich & List, 2007; Dokow & 
Holzman, 2010; both building on results of Klaus Nehring 
and Clemens Puppe later published in Nehring & Puppe, 
2010). So, whether a proposition is socially believed 
must depend not only on the individual beliefs about 
this proposition but also on the individual beliefs about 
other propositions. This holistic nature of social beliefs 
makes social beliefs a more interesting and less trans- 
parent concept. The most famous holistic procedure to 
form social beliefs is the premise-based procedure (e.g., 
Dietrich & Mongin, 2010; Pettit, 2001). In our court 
example, this procedure determines the social belief 
on the propositions p and g (the premise propositions) 
through a majority vote on each of these propositions, 
thereafter logically deducing the social belief about r 
(the conclusion proposition). So r is socially believed if and 
only if p and q are both socially believed. This is holistic 
since the social belief about r is no longer determined 
only by the individual beliefs about r, potentially over- 
ruling a majority belief about r. But the premise-based 
procedure comes with its own problems; for instance, 
it presupposes that we can prioritize certain (premise) 
propositions over other (conclusion) propositions. Rival 
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holistic procedures are distance-based rules (e.g., Miller 
& Osherson, 2008), sequential rules (e.g., Dietrich, 2015; 
List, 2004), relevance-based rules (Dietrich, 2015), and 
“approximate majoritarian” rules (Nehring, Pivato, & 
Puppe, 2014). 

Probabilistic opinion pooling If we adopt the Bayesian 
paradigm, social beliefs should come in degrees rather than 
in binary yes/no form. What probabilities should society 
assign to propositions, given the probabilities assigned 
by the individuals? This is the so-called opinion-pooling 
problem—the probabilistic counterpart of the judgment- 
aggregation problem (for reviews, see Dietrich & List, 
2016; Genest & Zidek, 1986). Once beliefs are probabi- 
listic rather than binary, coherence of beliefs consists in 
respecting probabilistic principles like additivity, rather 
than logical principles like deductive closure. The pic- 
ture reverses entirely: propositionwise (“local”) aggrega- 
tion of beliefs no longer runs into trouble. The social 
probability of any proposition can simply be the aver- 
age (or a weighted average) of the individual probabili- 
ties, which guarantees coherent social beliefs as long 
as individual beliefs are coherent. Such “linear averag- 
ing” procedures have been characterized axiomatically 
by McConway (1981), Wagner (1982), Mongin (1995), 
and Dietrich and List (2017). Although social beliefs 
produced by linear averaging are coherent, they are not 
fully Bayesian. Bayesianism is usually taken to require 
two things: holding beliefs in probabilistic form (“proba- 
bilism”) and revising beliefs through Bayes’ rule (“condi- 
tionalization”), as explained in chapter 4.1 by Hajek and 
Staffel (this handbook). Social beliefs generated by linear 
averaging violate the second requirement: they fail to be 
revised via Bayes’ rule. Why? Assume some proposition 
is learnt, so that every individual revises his or her beliefs 
(via Bayes’ rule, assuming that individuals are Bayesian). 
If at any time the current social beliefs are the average of 
the current individual beliefs, then the post-information 
social beliefs usually differ from the initial social beliefs 
updated via Bayes’ rule. In short, this social belief revi- 
sion is non-Bayesian. Opinion pooling through geometric 
averaging repairs this flaw, that is, produces dynamically 
rational social beliefs. Here, social probabilities of worlds 
are obtained by a possibly weighted geometric average of 
individual probabilities (followed by a simple normal- 
ization ensuring that the total probability of worlds is 
1). Geometric opinion pooling is in fact the only way 
to guarantee “Bayesian” group beliefs of a suitably well- 
behaved sort, as has only recently been shown (Dietrich, 
2019; Russell, Hawthorne, & Buchak, 2015). Yet geomet- 
ric pooling has a different flaw, which linear pooling 
avoids: social probabilities are not robust to refining or 
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coarsening the algebra of propositions considered, mak- 
ing social beliefs description-sensitive. 

Aggregating evaluations—absolute versus ordinal approach 
Certain objects (e.g., wines, political candidates, holiday 
destinations) must be evaluated in terms of some criterion 
(e.g., moral value, well-being, aesthetic value, or size). Let 
this be an epistemic problem: evaluations express beliefs 
about facts of some kind—which is plausible for some 
evaluations (e.g., of size) and more controversial for others 
(e.g., of well-being). A first question is whether value is 
measured in absolute or ordinal terms. Consider the set 
X of objects evaluated. Ordinal evaluations are captured 
by a binary relation > on X, where “x >y” means that x 
is at least as valuable as, or ranks weakly above, y (with 
respect to the relevant criterion). Absolute evaluations are 
captured by a function assigning to each object in X a 
value or rank from a set V of possible values or ranks, 


for example, from {very good, good, .. .} (moral evalu- 
ation) or {beautiful, ugly, ...} (aesthetic evaluation) or 
{large, medium, . . .} (size evaluation) or a set of numbers 


(numerical evaluations, e.g., of size). What should the 
social evaluations of objects be, given the evaluations by 
the group members? 


* In the ordinal case, this problem is structurally the 
notorious preference-aggregation problem, reinterpret- 
ing preference relations as value-judgment relations. 
One of many proposals is to use pairwise majority 
voting: object x is socially ranked over object y if and 
only if more individuals rank x over y than y over x. 
This procedure can lead to social cycles: some object x 
is majority-ranked over another, y, which is majority- 
ranked over another, z, which is majority-ranked over x 
(Condorcet’s voting paradox). It generalizes into Arrow’s 
impossibility theorem (Arrow, 1963): very roughly, not 
just pairwise majority voting but any pairwise aggre- 
gation procedure is flawed. This impossibility can be 
escaped by using some “holistic” rather than pair- 
wise aggregation rule, for instance, the Borda rule or 
a distance-based rule. A different escape is to replace 
ordinal by absolute evaluations. 


+ The picture is brighter when aggregating absolute evalu- 
ations—an aggregation problem called the social grad- 
ing problem by Balinski and Laraki (2011). In case of just 
two possible values—“approved” and “nonapproved”— 
the most natural procedure is “approval voting”: an 
object is socially approved if the number of individuals 
approving it is at least as high as for each other object, 
and socially nonapproved otherwise (Brams & Fish- 
burn, 2007). Balinski and Laraki generalize this proce- 
dure to any set of values linearly ranked from “highest” 
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to “lowest,” for example, {good, medium, bad}, where 
“good” ranks above “medium,” which ranks above 
“bad.” To socially evaluate an object x, first order the 
individuals such that the first individual evaluates 
x at least as highly as the second, the second at least 
as highly as the third, and so on. The social value of 
x is the value assigned to x by the middle individual 
in that order (assuming an odd number of individuals 
for simplicity). So, if in a wine evaluation among three 
judges, some wine receives the evaluations (exquisite, 
exquisite, drinkable), then this wine is socially evalu- 
ated as exquisite. This so-called majority-judgment rule 
(one might have called it the “median rule”) has sev- 
eral appealing features (see Balinski & Laraki, 2011) but 
remains controversial within social choice theory with 
its ordinalist tradition. 


Aggregating evaluations can be regarded as a special 
case of judgment aggregation by using propositions of 
type “x ranks over y” (for options x and y in X), in the 
ordinal case, or “x has value or rank v” (for options x in 
X and values v in V), in the cardinal case. 


2.2. Making Social Decisions 

The procedures discussed above generate social beliefs 
of different sorts. This section turns to procedures for 
making other social decisions.® To stay within social 
epistemology (broadly construed), the decision in ques- 
tion is assumed to track some truth, in particular, can be 
“correct” or “incorrect” in some procedure-independent 
sense (see sections 1.2, 1.3, and 1.6). For example, a 
group might have to choose a member from a set K of 
social alternatives. K could contain just two alternatives, 
as when a court decides whether to convict or acquit 
the defendant. K could instead contain many alterna- 
tives, as when a court decides between different sen- 
tences. When using plurality rule, each individual votes 
for exactly one alternative in K, and society chooses the 
alternative receiving the highest number of votes (or one 
such alternative in case of a tie). Plurality rule reduces to 
simple majority rule in the two-alternative case. In the 
many-alternative case, plurality rule can lead to prob- 
lems: suppose alternative k* in K is the “correct” alter- 
native. Plurality rule will normally fail to select k* if K 
contains many alternatives similar to k* (“clones” of k*), 
because k* will tend to lose votes to its clones. Worse, 
plurality will usually not even select one of the “approxi- 
mately correct” clones of k*, because these clones will 
themselves tend to lose votes to similar alternatives. 
One response is to replace plurality rule with approval 
voting (Brams & Fishburn, 2007), in which individuals 
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can vote for any number of alternatives and the most 
often approved alternative wins; here the correct alterna- 
tive will not tend to lose votes to its clones because indi- 
viduals can approve many alternatives. Another response 
is to base the social choice on how each individual ranks 
alternatives from “best” to “worst.” Here we aggregate 
individual rankings into a socially winning alternative 
(or set of winning alternatives, if ties are permitted). 
This aggregation problem differs structurally from the 
preference-aggregation problem mentioned above, in 
that social outputs are winning alternatives rather than 
rankings of alternatives. Procedures with such outputs are 
called social-choice rules rather than preference-aggregation 
rules. Nonetheless, the logical difficulties surrounding 
preference aggregation—illustrated by Condorcet’s voting 
paradox and culminating in Arrow’s (1963) theorem— 
reemerge for social-choice procedures. For instance, there 
may not exist any “Condorcet winner,” an alternative 
that is majority-ranked over each other alternative—a 
problem closely related to Condorcet’s voting paradox. 


2.3 Jury Theorems: Formal Truth-Tracking Arguments 
Do aggregation procedures like those just introduced suc- 
ceed in tracking the truth? Jury theorems provide formal 
“wisdom of crowds” arguments, to the effect that appro- 
priate procedures—typically majoritarian procedures— 
tend to generate correct social beliefs or other social 
decisions. Jury theorems can be powerful instruments— 
but they can also convey a misleading message when 
applied wrongly, because their optimistic conclusions 
may rely on misguided assumptions. The simplest jury 
theorem is Condorcet’s (1785) jury theorem. It assumes 
a majority vote between two social alternatives of which 
exactly one is “correct” or “better,” for instance, whether 
or not to convict or acquit a defendant. Jury theorems 
address the effect of increasing the size of the group. They 
typically conclude that “groups are wise” in the sense that 
one or both of two controversial hypotheses holds: 


* The growing-reliability hypothesis: Larger groups are 
better truth-trackers. That is, they are more likely to 
reach the correct outcome (in most jury theorems: by 
majority) than smaller groups or single individuals. 

* The infallibility hypothesis: Huge groups are infallible 
truth-trackers. That is, the likelihood of a correct out- 
come tends to full certainty as the group becomes 
larger and larger (Dietrich & Spiekermann, 2020). 


Different jury theorems differ in which of the two 
conclusions they reach and which premises they rest on. 
For instance, Condorcet’s theorem reaches both conclu- 
sions, based on two simple premises: an independence 
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assumption, whereby voters have independent probabil- 
ities of voting for the correct alternative, and a competence 
assumption, whereby these correct-voting probabilities 
exceed % and are the same for each voter. The infallibil- 
ity hypothesis then follows easily using the law of large 
numbers and the growing-reliability hypothesis follows 
from a more sophisticated combinatorial argument. The 
infallibility hypothesis, however, is overly optimistic 
and has left many with the (correct) impression that 
“something” must be wrong with those jury theorems 
that reach that conclusion, although there is confusion 
about the source of the problem. Some blame Con- 
dorcet’s unrealistic competence assumption. However, 
the implausible infallibility conclusion remains if com- 
petence can vary across individuals as long as it exceeds 
Y on average (Dietrich, 2008; Owen, Grofman, & Feld, 
1989). The real problem is the independence assump- 
tion. Although this assumption can also be weakened 
without losing infallibility (e.g., Pivato, 2017, and the 
literature reviewed in Dietrich & Spiekermann, 2020), 
plausible weakenings of independence make infallibility 
collapse (Dietrich, 2008; Dietrich & List, 2004). 

An important source of independence failure are com- 
mon causes affecting voters: common evidence, common 
paradigms or perspectives, and even noninformational 
common causes such as room temperature (Dietrich & 
Spiekermann, 2013). The limited nature of available evi- 
dence places objective bounds on the reliability of major- 
ity judgments, which cannot be miraculously overcome 
by including more and more voters exposed to the same 
limited evidence. But it would be hasty to dismiss jury the- 
orems. Although infallibility is unrealistic, the growing- 
reliability conclusion can be saved: that conclusion is 
reached by a jury theorem that revises Condorcet’s inde- 
pendence and competence assumptions in defensible 
ways (Dietrich & Spiekermann, 2013, 2020). In other work, 
Condorcet’s jury theorem has been extended beyond 
binary social decisions, such as choices between many 
alternatives via plurality voting (List & Goodin, 2001). In 
fact, there is a jury theorem for almost any standard vot- 
ing rule, including approval voting, distance-based rules, 
the Borda rule, and other scoring rules (Pivato, 2017). The 
question is thus not whether jury theorems “exist” to 
defend a given aggregation rule but whether those theo- 
rems use acceptable premises.’ 


3. Institutions beyond Mere Aggregation 
Besides voting procedures, there are many other institu- 


tional and social processes for promoting social knowl- 
edge and correct social decisions. This section gives three 
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important examples: deliberation, distributed search, and 
prediction markets. 


3.1 Deliberation 

The procedures discussed so far emphasize information 
aggregation. Deliberation, by contrast, emphasizes the 
exchange of reasons, prior to or instead of aggregation. 
Is deliberation truth-conducive? Several epistemic effects 
are at work: (i) deliberation can make private information 
or reasons public and allow participants to incorporate 
information and reasons of others; (ii) critical exchanges 
can eliminate bad reasons or inconsistencies and make 
good reasons and consistent viewpoints stand out more; 
(iii) deliberation might eliminate biases and reduce the 
influence of opinion leaders and other common causes; 
(iv) while deliberating, the decision problem can evolve, 
as new options come on the agenda and existing issues 
are reframed (Goodin & Spiekermann, 2018, chapter 9); 
and (v) deliberation can induce a meta-consensus about 
the structure of the decision space (Dryzek & List, 2003; 
cf. Dietrich & List, 2010; List, 2002). 

Extensive empirical research on “deliberative polls” 
suggests many benefits of deliberation (e.g., Fishkin, 2018, 
and much of his other work). It is particularly striking that 
opportunities for citizens to deliberate with decision mak- 
ers increase political knowledge (Esterling, Neblo, & Lazer, 
2011), although it is hard to disentangle whether this is 
an effect of the deliberative process or of opportunities for 
influence. However, there are also warnings that delibera- 
tion can lead to increased polarization (Sunstein, 2002). A 
good overview of the deliberation literature is Bachtiger, 
Dryzek, Mansbridge, and Warren (2018); Landemore 
(2017) is helpful for epistemic aspects. 


3.2 Distributed Search 

Some problems are best tackled by dividing epistemic 
labor. Scientific research, for example, progresses not 
because all scientists work on the same problems, with 
the same frameworks, theories, evidence, and so on, but 
because of a competitive diversity of approaches, with 
incentive structures rewarding originality while ensur- 
ing that resources are not too concentrated. In a similar 
vein, some have argued that federalism is an epistemically 
advantageous political system because different political 
subunits can search for and try out different solutions 
to a problem, an advantage that Judge Brandeis referred 
to as the “laboratory” of federalism (New State Ice Co. v. 
Liebmann, 1932). Distributed search is particularly suit- 
able when (i) the search domain is large (e.g., many rival 
theories could be tested); (ii) identifying correct choices 
requires effort, for example, testing a theory (to an extent 
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sufficient for acceptance or rejection) is an expensive and 
lengthy process; and (iii) search in different locations 
causes useful by-products (e.g., new discoveries). 

In distributed search, individual rationality need not 
be conducive to social rationality. In scientific research, 
for example, a bad incentive system would render it indi- 
vidually rational for each scientist to pursue research 
in novel areas receiving most attention and funding. 
However, that will likely lead to inefficient allocation of 
resources while valuable subfields are left unexplored, 
hampering overall scientific progress. A socially rational 
division of scientific labor would promote more diversity, 
more replication, and work on important projects with 
less imminent rewards. The influential work by Kitcher 
(1993) and Strevens (2003) shows how a mix of incentive 
schemes helps modern science manage this tension, often 
with success. The importance of diversity and the tension 
between individual and social rationality have been mod- 
eled in different ways (e.g., Hong & Page, 2012; Weisberg 
& Muldoon, 2009; Zollman, 2010; see also chapter 14.1 
by Andersen & Andersen, this handbook). The theoretical 
results suggest that a socially irrational division of epis- 
temic labor can emerge from individual rationality. 


3.3. Prediction Markets 
Prediction markets are financial markets from which pre- 
dictions of contingent events, such as election outcomes, 
emerge. The securities traded on these markets are futures 
on the occurrence of these events. The mechanisms 
behind prediction markets are aggregation and incentiv- 
ization (Mann, 2016; Wolfers & Zitzewitz, 2004). We have 
already seen that aggregation through voting can track 
the truth. The clever addition of prediction markets is the 
incentive created by gains when the future is under- or 
overpriced. The competitive setting motivates participants 
to try to outwit others by finding more information. The 
prediction market is “successful” when it predicts—more 
precisely, attaches high price (probability) to—events 
that eventually happen. Prediction markets have often 
successfully predicted political events (see, e.g., the Iowa 
Electronic Markets*), business events, sports events, and 
even scientific events (Arrow et al., 2008; Dreber et al., 
2015). However, they are not infallible, mostly because 
of inefficient markets, systematic mistakes of traders, and 
insufficient or misleading information. For example, the 
Iowa Prediction Markets wrongly predicted a “no” in the 
2016 UK “Brexit” referendum and a Clinton victory over 
Trump in the 2016 US presidential election.’ 

Prediction markets track the truth in an interestingly 
different way: while standard aggregation procedures 
transform explicit individual judgments into an explicit 
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group judgment or other decision, prediction markets 
transform entirely behavioral inputs (buying and selling) 
into market prices from which implicit, but precise, group 
judgments can be read off. Although individual judg- 
ments are revealed only indirectly through acts of buy- 
ing or selling, they are elicited reliably: while voting gives 
incentives to misrepresent, prediction markets do not 
normally provide such opportunities because counter- 
judgmental trading leads to losses. 


4. Some Sources of Social Epistemic Failure 


This chapter has already shown several reasons why 
groups can fail to form correct beliefs or other correct 
decisions: its members can fall prey to biases, be influ- 
enced by opinion leaders, or follow misleading evidence. 
This section briefly reviews some further sources of social 
epistemic failure. Most epistemic failures begin as indi- 
vidual failures, pertaining to epistemology in groups. But 
since individual beliefs shape group beliefs, individual 
failures become social failures and may even get ampli- 
fied through deliberation or aggregation. So they also 
pertain to epistemology of groups, the main focus of this 
chapter. An exception is strategic voting, which involves 
no false individual beliefs but may lead to social epis- 
temic failures. 


4.1 Strategic Voting 

Even if voters care for nothing but correctness of social 
beliefs or decisions more broadly, they can have strategic 
incentives to misrepresent their views—which is surpris- 
ing given the absence of conflicting goals. For instance, 
a juror may vote for convicting the defendant, although 
her private information suggests innocence, as she rea- 
sons strategically that this makes the jury’s (majority or 
supermajority) decision more likely correct, despite mak- 
ing her own vote less likely correct (see Austen-Smith 
& Banks, 1996; for details of the strategic rationale, see 
Dietrich & Spiekermann, 2020). So individual and social 
rationality can conflict even when, unlike in a prisoner’s 
dilemma or tragedy of the commons, individual and 
social interests are perfectly aligned. 

It is, however, important to put things into perspec- 
tive. First, it is debatable whether such strategizing is 
socially harmful: while strategic voters systematically 
ignore private information (leading to underinformed 
social outcomes), they do so in order to improve social 
outcomes. The harmfulness of strategizing depends on 
the models and assumptions used. Second, voters often 
have perfectly rational grounds for voting truthfully, as 
they may care about expressing their own view instead 
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of (or in addition to) caring about correctness of the 
voting outcome, a phenomenon usually ignored in the 
strategic voting literature with its consequentialist ori- 
entation (Dietrich & Spiekermann, 2020). Paradoxically, 
voters stop strategizing because they stop (not begin) 
sharing the same goal of correct outcomes. 


4.2 Social Pressure 

A potent cause of epistemic failure is the desire to con- 
form with perceived expectations or norms, to avoid 
conflict or cognitive dissonance, to avoid mistakes in 
public, or to please peers. For example, group delibera- 
tion is likely to overemphasize information that was 
already widely accepted prior to deliberation and under- 
emphasize information that is sparsely spread and yet 
crucial for a correct decision. Such an underemphasis 
can happen not just because there are fewer people who 
hold the information in the first place but also because 
those who hold it stay silent, predicting, perhaps cor- 
rectly, that the information will be controversial (Gigone 
& Hastie, 1993; Stasser & Titus, 2003). Related effects of 
social pressure are the amplification of individual errors 
in deliberation and the possible enforcement of extreme 
positions—two deliberation failures (Sunstein & Hastie, 
2015a, 2015b). 


4.3. Motivated Cognition 

The desire to fit into an identity group or comply with 
social norms can bias the acquisition and cognitive pro- 
cessing of factual information in the first place (e.g., Spieker- 
mann & Weiss, 2016). For example, Kahan et al. (2012) 
describe how different political-cultural backgrounds cre- 
ate different pressures to seek or avoid information about 
climate change and to process such information in biased 
ways. Kahan et al.’s left-liberal subjects believe that climate 
change is a problem, right-libertarians much less so. Scien- 
tific or numeracy skills further increase this divergence, 
suggesting that better understanding leads to even more 
biased processing. 


4.4 Epistemic Injustice 

Someone suffers epistemic injustice if she is “wronged 
specifically in her capacity as a knower” (Fricker, 2007, 
p. 20). Fricker distinguishes between testimonial and her- 
meneutic injustice. Testimonial injustice is experienced if 
testimony is discounted because of an identity prejudice; 
for example, the evidence of a person of minority back- 
ground is dismissed because of their background. Herme- 
neutic injustice can arise if oppressed individuals or groups 
do not have adequate concepts to conceive of, make 
sense of, or communicate their experiences. For example, 
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Fricker explains that victims of sexual harassment found it 
difficult to understand their experience and explain it to 
others before the term “sexual harassment” was coined. 
Testimonial and hermeneutic injustice undermine the 
ability of individuals and ultimately the group to generate 
correct beliefs or other correct decisions. 


4.5 Epistemic Skepticism and Nihilism 

A more fundamental threat to epistemic success is the 
refusal to take a truth-seeking attitude. Epistemic skepticism 
is the belief that there is no truth to be tracked or that access 
to the truth is impossible. It can be distinguished from epis- 
temic nihilism, the view that truth does not matter, regard- 
less of whether or not it exists and is accessible. Whereas 
skepticism is a respectable epistemological position, nihil- 
ism is an epistemologically and morally dubious attitude. 
However, genuine nihilists will by definition not care about 
whether their attitude is defensible, which makes nihilism 
particularly immune to arguments and hard to fight against. 
Harry Frankfurt (2005) has suggested that the lack of care 
for the truth is one aspect of “bullshitting,” a lack of com- 
mitment to the truth and epistemic norms of truthfulness 
(cf. Cassam, 2019, especially chapter 4). The so-called fake 
news phenomenon seems fueled by epistemic nihilism, 
although the debate about this is still in its infancy (e.g., 
Goodin & Spiekermann, 2018, chapter 21; Mukerji, 2018; 
cf. Rini, 2017). The skepticism or nihilism of individuals 
can scale up to the group level, undermining the truth- 
tracking ability of media organizations, public forums, or 
the political system as a whole. 
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Notes 


1. This volume touches upon other issues of formal epistemology 
in chapter 2.6 by van Benthem, Liu, and Smets; chapter 5.2 by 
Rott; chapter 5.3 by Kern-Isberner, Skovgaard-Olsen, and Spohn; 
chapter 5.5 by Hahn and Collins; and chapter 5.6 by Woods. 


2. By using “social decision” broadly to encompass social 
beliefs we do not assume that group beliefs (let alone individual 
beliefs) always are under direct voluntary control. We set aside 
whether groups have such control, and what it would mean. 


3. If groups are construed as intentional agents, they also hold 
non-belief attitudes like desires, preferences, or intentions. This 
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immediately raises the question of whether holding such other 
attitudes is also “correct” or “incorrect” in some relevant sense 
(possibly just for certain attitudes, e.g., for Scanlon’s judgment- 
sensitive attitudes; Scanlon, 1998, p. 20). The correctness of 
holding, say, an intention might depend on whether its con- 
tent is morally permissible. For instance, it might be correct for 
a firm to intend to maximize shareholder value if maximizing 
it is morally permissible. Like social beliefs and correctness-apt 
actions, correctness-apt social attitudes might be regarded as 
a matter of social epistemology (broadly construed). Whether 
or not one adopts this inclusive view, some of the technical 
machinery of social epistemology—like jury theorems—can be 
used to study whether group attitudes become correct. 


4. A hybrid position is to deny normative facts but to grant 
that normative judgments can be correct or incorrect as acts 
(on other grounds than the truth or falseness of their con- 
tents). This is still compatible with social epistemology broadly 
construed (cf. section 1.2). 


5. What (formal) rationality means precisely depends on the 
type of belief (or other attitude or decision) considered and 
can be fleshed out differently. Binary yes—no beliefs are often 
required to be consistent and deductively closed. Probabilistic 
beliefs are often required to satisfy Kolmogorov’s axioms. Ordi- 
nal betterness judgments over some choice options are often 
required to be transitive and perhaps complete. 


6. The social beliefs or judgments discussed in section 2.1 
often already generate social actions: society can choose what 
it believes to be best (ordinally) or what it approves or rates 
highest (absolutely) or what is recommended by its judgments, 
and so on. The current section, however, investigates proce- 
dures that generate social actions directly, rather than generat- 
ing beliefs or judgments supporting actions. 


7. While standard jury theorems address the epistemic perfor- 
mance of a given procedure (usually majority voting), one may 
alternatively search for the epistemically optimal procedure, 
thereby accounting for differential competence across indi- 
viduals (Ben-Yashar & Nitzan, 1997; Dietrich, 2006). 


8. Available at https://iemweb.biz.uiowa.edu (last accessed 
April 17, 2019). 


9. The low probabilities attached to Trump or Brexit victories 
were, strictly speaking, not mistakes. They merely showed that 
prediction markets, like many pollsters, underestimated sup- 
port for Trump and Brexit. 
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10.2 Collective Rationality 


Hans Bernhard Schmid 


Summary 


“Collective rationality” is used in a variety of ways in the 
relevant literature, two of which stand out: (a) the ratio- 
nality of cooperation and coordination and (b) corporate 
rationality (or the rationality of group agents). These two 
conceptions are markedly different from each other in the 
issues they concern and in the concepts of collectivity and 
rationality they involve. Upon closer inspection, however, 
it turns out that from different sides, they converge on the 
same issue, namely, the capacity for rational joint action. 


1. Rational Joint Action, Cooperation, 
and Corporation 


In the first of the two versions in which collective ratio- 
nality is currently discussed, it is a relational feature of 
choices of individuals. The paradigmatic case of this 
sense of collective rationality is mutual cooperation in 
a two-person prisoner’s dilemma. Individual choices are 
collectively rational if they realize a mutually cooperative 
outcome (an outcome that is better for both than mutual 
defection), even though individual strategic reasoning 
recommends mutual defection. Defection is individually 
rational (in a strategic sense) because from the point of 
view of each of the participating individuals, choosing 
to defect realizes a better state no matter whether the 
other cooperates or defects. Among strategically ratio- 
nal individuals, mutual defection is thus the expected 
outcome—which seems a bit foolish given that they 
could have realized an outcome that is better from both 
participants’ points of view. Collective rationality in this 
sense is thus defined in contradistinction against (a stra- 
tegic conception of) individual rationality. Conversely, 
there is an air of individual irrationality about individ- 
ual choices that are collectively rational, even though in 
terms of efficiency, the participants are ultimately better 
off through mutual cooperation. Collective rationality 
in this sense is a matter of the rationality of cooperation 
(although, as will be argued below, coordination might be 


a better paradigm to study this first conception of col- 
lective rationality). The rationality of cooperation and 
coordination matters because it explains the provision 
of common and public goods (cooperation) and the role 
of conventions in action (coordination). 

In the second salient sense of the word, collective 
rationality is an attribute of collective or group agents, 
their attitudes, and their actions. It is the feature in virtue 
of which collective or group agents (such as political par- 
ties, universities, or business corporations) are consistent 
in their attitudes and actions over more or less complex 
interdependent decisions. In this second sense, collective 
rationality is defined in analogy to (rather than in con- 
tradistinction against) individual rationality. Just as ratio- 
nal individual agency needs some degree of consistency 
across issues, it seems that at least some collectives need 
to ensure that their collective views and decisions are 
mutually consistent too. The problem of collective ratio- 
nality in this second sense, as discussed in the recent 
literature, is that it is not guaranteed by the rationality 
of individuals. No plausible fixed aggregation procedure 
robustly ensures the rationality of collective decisions 
among rational individuals. Achieving collective ratio- 
nality is thus an extra achievement necessary for orga- 
nized social life—the “collectivization of reason,” 
is sometimes called in the literature. This second con- 
ception of collective rationality matters insofar as the 
credibility and trustworthiness of group agents such as 
institutions and organizations depend on their consis- 
tency in their attitudes and actions. 

These two conceptions of collective rationality—the 
rationality of coordination and cooperation, as well as 
corporate rationality—thus seem to be markedly dif- 
ferent in their targets and approaches. Perhaps more 
important, they radically differ both with regard to the 
idea of rationality as well as with regard to the notion 
of collectivity they entail. The rationality of coopera- 
tion between individuals is a matter of efficiency, and 
it involves collectivity in the sense that it is a relational 
feature of participant individual choices. By contrast, the 
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rationality of collective agents (or corporate rationality) 
is a matter of consistency, and it involves collectivity in 
terms of agency that is collective rather than individual. A 
further distinction is that while the rationality of coop- 
eration is defined in contradistinction from individual 
rationality, corporate rationality is defined in analogy to 
individual rationality. 

After overviews over each of these two debates (sec- 
tions 2 and 3), this chapter makes a suggestion con- 
cerning how, despite their differences, they might hang 
together (section 4). The claim is that the feature that 
rationalizes cooperation (and coordination) between 
individuals is also the basic force that drives toward 
coherence between group attitudes. Both conceptions 
thus converge on the capacity to think and act together, 
from a joint point of view, and to do so consistently and 
effectively. This capacity is currently mostly discussed 
under labels such as “collective intentionality.” 


2. The Rationality of Coordination and Cooperation 


In the first sense of the word, it is sometimes claimed in the 
literature that human social activity cannot be based on 
individual rationality alone but has to be complemented 
with—or perhaps partially replaced by—collective ratio- 
nality. The relation between individual and collective 
rationality is thought to be difficult because the two can— 
and often do—come into conflict with each other. The 
basic problem has been noticed at least since Hobbes (cf. 
Gauthier, 1979; with a bit of good will, one could perhaps 
reconstruct a passage in Plato’s Republic [359a] along simi- 
lar lines). In a game-theoretic framework (cf. chapter 9.1 
by Albert & Kliemt, this handbook), the issue at stake has 
been at the focus of social theory since the 1950s under 
the name of “the prisoner’s dilemma” (Kuhn, 2017; an 
alternative—and equally correct—spelling is “prisoners’ 
dilemma”). 

The prisoner’s dilemma (PD) is a situation of interde- 
pendent choices between two available options (choices, 
or strategies), c and d, where in the case of two partici- 
pants A and B, the outcome if A d’s and B c’s realizes best 
what A values and worst what B values (and vice versa 
for B’s d’ing and A’s c’ing), and where mutual c’ing real- 
izes what A and B value better than mutual d’ing. The PD 
does not presuppose selfish preferences or values. It can 
arise between altruists, too (cf. Tilley, 1991), although it 
is usually illustrated with the example of self-interested 
(and mutually disinterested) participants and has its 
most important applications in this context. The reason 
why the system of evaluations that is the PD is so impor- 
tant in the debate is that it captures the basic structure 
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of the problem of common and public goods—that is, 
goods that are nonexcludable and therefore invite free- 
riding (the “tragedy of the commons”). The dilemma, as 
it is usually conceived of in the literature, is “a conflict 
between individual and collective rationality” (Orbell 
& Wilson, 1979, p. 411). It is claimed that in situations 
such as the PD, “individually rational” choice results in 
“collective irrationality” (e.g., Sorensen, 2004, p. 265). 
In this sense, collective rationality is thus a property of a 
set of individual choices that are such that the individu- 
als reap the benefits of mutual cooperation in the face of 
temptation to defect. Because of this structure of incen- 
tives, there is, however, an air of irrationality about coop- 
erative choices. Standard strategic rationality demands of 
individuals that if there is an option that realizes a better 
outcome (measured by one’s own desires or values) no 
matter what the relevant others do, one should choose 
that strategy. The cooperative choice is strictly dominated 
by the defective choice—to choose the cooperative strat- 
egy realizes a worse outcome no matter what the other 
does. This recommends defection in PDs as the rational 
choice (and since this is true for both participants, mutual 
defection is the only Nash equilibrium). 

It is hard to accept from an intuitive point of view, 
however, that it should be rational for agents with com- 
mon knowledge of rationality as well as common knowl- 
edge of the payoff structure that they should knowingly 
realize a combination of choices that will make each of 
them worse off than another available outcome. To put 
this point in the relevant jargon: the problem is that 
mutual defection is the only Pareto-inefficient outcome 
of a PD. It seems counterintuitive to say that it is rational 
to be inefficient. Agents whose rationality is only strate- 
gic, but foreseeably inefficient, thus appear to be “ratio- 
nal fools” (Sen, 1977). 

Although not all relevant authors use the term “collec- 
tive rationality” in this context, the project of safeguard- 
ing (combinations of) cooperative individual choices 
in PDs from the allegation of irrationality has featured 
prominently in the agenda of social theory and social 
philosophy, especially since the 1970s (for an extended 
discussion see Townley, 2008, chapter 9). The reason 
why this is an important issue is that there are limits to 
aligning individual strategic rationality with the produc- 
tion of public goods via institutional design. To some 
degree, social order depends on individual cooperative- 
mindedness, and among likeminded people, cooperative- 
mindedness is efficient—and is thus not simply irrational, 
although the rationality involved here is somewhat in 
tension with practical rationality of the standard indi- 
vidual (strategic) kind. 


Collective Rationality 


Most prominent among the more recent attempts 
is Paul Weirich’s (2009) conception of collective ratio- 
nality. Weirich defends participant cooperative choices 
from the allegation of irrationality by arguing that self- 
support rather than utility maximization is the standard 
to which it should be held; a combination of coopera- 
tive choices is self-supporting. In this vein, Weirich 
argues that it is a sufficient condition for the rationality 
of a combination of choices (a “group act,” in Weirich’s 
terms) that its individual members’ acts are rational (in 
his revised sense). 

Another conception of collective rationality in this 
sense is Christopher McMahon’s (2001). According to 
his view, it is (collectively) rational to contribute to a 
“cooperative scheme” if the “outcome of the scheme, 
when one’s contribution is added to the others that will 
actually be made, exceeds the value to one of the nonco- 
operative outcome” (McMahon, 2001, pp. 21-22). 

This conception of collective rationality is, in turn, 
similar to David Gauthier’s (1986) classical concept of 
constrained maximization (cf. chapter 12.1 by Fehige & 
Wessels, this handbook, as well as a considerable num- 
ber of similar attempts to vindicate the rationality of 
cooperative choices). However, Gauthier avoids using 
the term “collective rationality.” Indeed, it is far from 
obvious why the rationality of cooperative choices, as 
conceived in this debate, should be collective rather than 
individual. After all, it is the individuals who profit from 
mutual cooperation (or suffer from mutual defection), 
not some collective entity such as “the team.” In this 
regard, one might even think that rather than a dilemma 
(i.e., a choice between equally problematic alternatives), 
the PD is really a paradox (i.e., a self-contradiction; an 
example for this interpretation is Davis, 1977) or per- 
haps a tragedy (a contradiction between intention and 
effect; cf. Tilley, 1994). Strategic rationality (that is aimed 
at maximizing expected utility) recommends a choice of 
which it is known that it fails to maximize utility among 
strategically rational individuals. The paradox of the PD 
is thus, in this view, that “so-called irrational players .. . 
will both fare much better than so-called rational ones” 
(Luce & Raiffa, 1957, p. 96; the fact that conversely, 
“rational” players fare worse than irrational ones might 
be called the tragedy of the PD). 

Although the efficiency of mutual cooperation is cer- 
tainly a good ground to claim some sort of rationality 
for the respective choices, considered in combination, 
the sense in which the rationality of cooperation is col- 
lective rather than individual certainly needs clarifica- 
tion. From this perspective, rather than pitting collective 
rationality against individual rationality, the PD seems 
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to separate efficiency from strategy dominance, and it is 
not obvious how exactly the rationality of the efficient 
choice involves collectivity other than in the rather 
obvious sense that it takes at least two to realize a mutu- 
ally cooperative result. 

In this regard, the debate on coordination is illuminat- 
ing. While cooperation in PDs requires the participant 
to choose against their strategic self-interest, there is no 
such conflict in pure coordination problems (for a dis- 
cussion of coordination and cooperation in game theory 
and behavioral economics see Butler, 2012). In situa- 
tions of pure coordination, there is no dominant strat- 
egy for the participants; what is rational for them to do, 
individually, depends on what they expect the others to 
choose (of whom it is known that they are in the same 
situation). Where none of the available equilibria is 
better for all of the participants, the problem is usually 
very effectively regulated by conventions (or salience) 
in real life, and although it seems intuitively obvious 
that it is rational for the participants to solve coordina- 
tion problems by choosing the conventional option, it is 
impossible to sustain this intuition in terms of individ- 
ual strategic reasoning (where mutual expectations are 
interdependent rather than fixed or convergent). Classi- 
cal rational choice theory cannot recommend the choice 
of the “salient” strategy as the one rational choice. Even 
more counterintuitive is the recommendation of classi- 
cal rational choice theory in the case where one coor- 
dination equilibrium is better for all than the other(s) 
(such as in the Hi-Lo game, where the choice of “Hi” by 
both participants realizes a better result for both than the 
choice of “Lo” by both partners; “Lo”/“Lo” is still bet- 
ter for both than “Hi”/“Lo” or “Lo”/“Hi”). In this case, 
it seems intuitively very clear that common knowledge 
of rationality and of the utilities rationalizes the accord- 
ing choice (i.e., “Hi”). Choosing “Lo” would be irrational. 
However, this intuition is, again, not sustained by clas- 
sical rational choice theory. (Especially counterintuitive 
is the mixed-strategy equilibrium strategy with a higher 
probability of the strategy that has the worse coordina- 
tion equilibrium as a potential outcome, i.e., “Lo.” The 
mixed-strategy equilibrium thus equalizes the expected 
utilities of the two strategies.) 

The general structure of interdependent expectations 
in situations of coordination (“multiple contingency”) 
leads such social theorists as Talcott Parsons and Niklas 
Luhmann to think that social science should be framed 
in systems-theoretic terms rather than in terms of rational 
agency—a strike through the Gordian knot of rationaliz- 
ing coordination that seems to have lost a great deal of its 
appeal in social science over the past couple of decades 
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(Schmid, 2011; for the use of game theory in the social sci- 
ences, cf. chapter 10.4 by Raub, this handbook). In recent 
action and decision theory, the issue has been taken up, 
and a series of theories of the rationality of coordinated 
choices has been developed. In this context, the collectiv- 
ity of the rationality in question has received particular 
attention. Most prominently, Robert Sugden (1993) and 
Michael Bacharach (2006) have suggested that the ratio- 
nality of coordinated choices is collective in the sense of 
a certain kind, or mode, of reasoning. Rather than rea- 
soning strategically from their individual points of view, 
the participants engage in team reasoning. They recognize 
situations of coordination (such as the paradigmatic Hi- 
Lo game) as situations in which “the team”—“we,” from 
the agents’ perspective—has to choose, and this rational- 
izes their contribution to the best collective choice. The 
way this matters to the prisoner’s dilemma is that aversion 
against exploiting others (or being exploited by them) 
transforms a PD into a Hi-Lo game with mutual coop- 
eration as the better equilibrium (Gold & Sugden, 2007, 
pp. 288ff.). The way in which this relates to the open ques- 
tion of the collectivity of the collective rationality in ratio- 
nal team reasoning is that it is “as a team” (rather than 
“as individuals”) that the participants reason. As Susan 
Hurley (2003) puts it in her pithy title, “The limits of indi- 
vidualism are not the limits of rationality.” It is widely 
accepted in the relevant literature that this should not be 
taken as saying that it is simply another rational choice 
for individuals to identify with teams and reason from a 
team perspective, as this would lead back into the prob- 
lem of interdependent expectations (it is rational if the 
others choose to identify with the team too, etc.). At this 
point, Elizabeth Anderson’s “priority of identity” princi- 
ple is relevant: before the question of rational choice can 
be answered, the question of the identity of the agent has 
to be settled, and very few agents are “merely individuals” 
but rather members of a wide variety of groups (Ander- 
son, 2001, p. 30). However, the claim that the—singular 
or plural—identity of the chooser is always a matter of a 
subpersonal framing process that is beyond the scope of 
reasoning is equally dissatisfying as the “rational identity 
choice” view. Clearly, we do sometimes reason about from 
which point of view to approach a decision, such as in 
the case of role conflicts, and it is not clear that we should 
take a “purely individual” point of view as the default. 


3. The Rationality of Corporations 
In a second and apparently quite different sense of the 


word, “collective rationality” is used in the literature 
for a particular relation between beliefs, desires, and 
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perhaps intentions had by a collection, or group. Col- 
lective rationality is the feature in virtue of which there 
are “consistent and complete group attitudes towards 
the propositions on the agenda” (List & Pettit, 2011, 
p. 49; cf. chapter 10.1 by Dietrich & Spiekermann, this 
handbook). In this sense, the very idea of collective 
rationality is predicated on the assumption that besides 
individuals, collectives, too, can have an agenda, that is, 
that they, too, can be epistemic and practical agents (and 
be usefully treated as such in social science) and thus be 
rational. This assumption has received increasing sup- 
port in the recent literature, most notably from Chris- 
tian List and Philip Pettit (2011). This marks a decided 
departure from the widespread older methodological 
precept in social science and social philosophy to treat 
only individuals (and not collectives) as agents and thus 
as (potentially) rational. 

On this line, collective rationality is approached as 
an aggregation problem: how can individual judgments 
(preferences, beliefs, or desires) be aggregated in such a 
way as to yield a consistent collective standpoint across 
interdependent issues? It turns out on this line of research 
that no simple nondictatorial aggregation rule ensures col- 
lective rationality in a robust way (i.e., across a larger num- 
ber of interdependent decisions)—and that if we manage 
to be robustly rational (consistent in our joint position) 
as the teams and organizations we are, we cannot achieve 
this through mechanical application of simple judgment 
aggregation procedures (e.g., majority voting) alone. 

This second concept of collective rationality is best 
introduced indirectly. For in the 20th-century litera- 
ture in economics and other social sciences, collective 
rationality—in the sense of rational group agency—has 
repeatedly and prominently been declared to be neither 
possible nor even desirable. A particularly clear statement 
to this effect is in the chapter on individual and collective 
rationality in James M. Buchanan’s (1962/1999) Calculus 
of Consent: 


It is difficult to understand why group decisions should 
be directed toward the achievement of any specific end or 
goal. Under the individualistic postulates, group decisions 
represent outcomes of certain agreed-upon rules for choice 
after the separate individual choices are fed into the process. 
There seems to be no reason why we should expect these 
final outcomes to exhibit any sense of order which might, 
under certain definitions of rationality, be said to reflect 
rational social action. Nor is there reason to suggest that 
rationality, even if it could be achieved through appropri- 
ate modification of the rules, would be “desirable.” Rational 
social action, in this sense, would seem to be neither a posi- 
tive prediction of the results that might emerge from group 
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activity nor a normative criterion against which decision- 
making rules may be “socially” ordered. (pp. 31-32) 


In this view, the idea of collective rationality appears 
simply as an “illegitimate transfer from the individual 
to society” (Arrow, 1963, p. 120; cf. Suzumura, 1983, 
p. 94). According to Buchanan’s “individualistic postu- 
late,” only individuals have a will and values and make 
choices. Why deny collectives such features and abili- 
ties? The worry behind the individualistic position seems 
to be that collective agency demotes the participating indi- 
viduals from their place as proper agents to mere “organs” 
of the collective organism (cf. Buchanan, 1962/1999, 
pp. 11ff.). This worry is intimately connected to the his- 
tory of methodological individualism (Heath, 2015; 
Udehn, 2001). Buchanan’s views on collective rationality 
thus reflect Max Weber’s (1922/1968) canonical state- 
ments and echo Joseph Alois Schumpeter’s (1908/2010, 
chapter 6) view—with the remarkable difference that 
both of these classical methodological individualists saw 
the “individualistic principle” (Schumpeter) as a method- 
ological precept for the particular type or branch of social 
science they envisaged, and they explicitly allowed for 
conceptions of “societal values” (Schumpeter) or “collec- 
tive agents” (Weber) for other social sciences. 

The most influential challenge to Buchanan’s ban 
on collective rationality comes from Kenneth J. Arrow: 
“Collective rationality in the social choice mechanism 
is not...an illegitimate transfer from the individual 
to society, but an important attribute of a genuinely 
democratic system capable of full adaptation to vary- 
ing environments” (Arrow, 1963, p. 120). In particular, 
collective rationality is the remedy against “democratic 
paralysis”—“a failure to act due not to a desire for inac- 
tion but an inability to agree on the proper action” 
(Arrow, 1963, p. 120). Thus, contrary to Buchanan’s 
view, collective rationality may well appear as desirable, 
and very desirable indeed. But is it possible? 

The way the issue has historically been approached 
is in terms of aggregation of individual preferences to a 
social preference, and the debate revolved around Con- 
dorcet’s famous paradox (see Gehrlein, 2006) as the 
central challenge to collective rationality from an aggre- 
gative point of view. Assume a group consisting of A, 
B, and C. A’s preference order is x>y>z, B’s is y>z>x, 
and C’s is z > x > y. Each individual member’s preferences 
are complete and transitive. However, if they vote pair- 
wise over the alternatives, it comes out that there is a 
majority of two for x > y, a majority of two for y > z, but 
also a majority of two for z>x. Thus, the group’s pref- 
erences are intransitive: the preference order is circular. 
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Their predicament, as a group, is thus such that based on 
their aggregation procedure according to the majority 
rule, they cannot agree on one option, but neither are 
they indifferent between the options. A further reason 
to think that this makes the group irrational is that they 
can now be money-pumped (cf. chapter 8.1 by Griine- 
Yanoff, this handbook): give them x for free, then offer 
them z, which they prefer over x and are thus willing to 
pay for, then make them pay for y, then x, then z again, 
and so on and so forth until they are broke. It seems 
intuitively irrational to have a structure of preferences 
that makes one vulnerable to such exploitation. 

Kenneth J. Arrow’s (1963) famous impossibility 
theorem—which can be seen as a generalization of the 
Condorcet theorem (List, 2011)—shows that collective 
rationality is incompatible with other plausible condi- 
tions of collective choice: unrestricted domain (there is 
a unique and complete ranking of collective choices no 
matter what the individual preferences are that go into 
the decision process), non-dictatorship (no individual 
has the power to always determine the group’s choice), 
independence of irrelevant alternatives (the social pref- 
erence between x and y should depend only on the indi- 
viduals’ preferences over x and y), and Pareto efficiency 
(if every participant individual prefers x over y, there is 
a collective preference of x over y). Many authors have 
searched for—and suggested—potential escape routes. 
Most prominent among these is Amartya Sen’s (1969) 
proposal of a weaker conception of collective rationality; 
however, this condition turned out to contain an oligar- 
chy (see Blair, Bordes, Kelly, & Suzumura, 1976). 

While Arrow’s impossibility theorem concerns prefer- 
ence aggregation, judgment aggregation is at the center of 
the more recent debate on the desirability and (im)pos- 
sibility of collective rationality. This development might 
be seen as a generalization (cf. Nehring, 2003), as a shift of 
focus (depending on whether or not one interprets prefer- 
ences as judgments—a preference of x over y can be inter- 
preted as the judgment that x is better than [or preferable 
to] y), or perhaps as not making much of a difference at 
all (if one interprets epistemic judgments as preferences 
over what to believe). Again, the point is that groups and 
other collective bodies often have to arrive at judgments 
concerning what is the case and what should be done, 
and they have to show some degree of consistency in 
their judgment in order to function as the group or col- 
lective body they are. 

For this new wave in the discussion on social choice, 
the discursive dilemma plays a similar role as did Con- 
dorcet’s paradox for the Arrovian line and has largely 
replaced the latter in the recent debate. The main point, 
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however, is the same: the combination of majority vot- 
ing rule and rationality of the participant individuals 
does not ensure collective rationality. Philip Pettit’s 
(2003) example for the discursive dilemma is a political 
party of three members that is unanimously commit- 
ted to the view that deficit spending is not a good idea. 
Now a series of policy decisions comes up. The party first 
votes on whether or not taxes should be increased—two 
of the three members vote “no,” so this is the party view. 
Then the issue of defense spending comes up—two of 
three members vote for an increase, one for a reduction, 
making “increase” the party line. Then the issue of other 
government spending comes up—again, increase has a 
two-thirds pro majority, with only one member voting 
for reduction. Given the party’s view on deficit spend- 
ing, the party is thus inconsistent. However, the partici- 
pant individuals may well be individually consistent and 
thus rational: one of the votes for an increase of defense 
spending and other spending comes from the member 
who voted for a tax increase, while the second vote for 
increased defense spending comes from a member who 
voted for a reduction of other spending, and the second 
vote for an increase of other spending comes from a 
member who judged that defense spending should be 
reduced, so that each of the participants individually 
respects the incompatibility of increased spending, non- 
increase of taxes, and rejection of deficit spending. 

Thus, the discursive dilemma again shows that the 
combination of individual rationality and majority voting 
does not ensure collective rationality. It is controversially 
discussed in the literature on the discursive dilemma, 
however, how much collective rationality really matters. 
For example, in the context of the question of democratic 
legitimacy, Fabienne Peter (2009) distinguishes reason- 
responsiveness from rational consistency (cf. chapter 2.1 
by Broome, this handbook) and argues that the latter is 
not in itself necessary for legitimacy: 


Is it really the case that, as a democratic collective, we ought 
to satisfy some requirements of collective rationality? Does 
the binding force of legitimate decision hinge on whether or 
not these decisions satisfy requirements of rationality? .. . 
In light of the distinction between reasons and rationality, 
it is possible to see that to deny that democratic legitimacy 
includes a rationality requirement does not entail that the 
demand to respect a democratic decision as legitimate does 
not depend on reasons. It is only to deny that there is inde- 
pendent normative force in the fact that collective decisions 
satisfy some conditions of consistency. (pp. 144-145) 


While this seems plausible as far as the question of 
democratic legitimacy is concerned, an issue of credibility 
and trustworthiness comes up when inconsistency is not 
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plausibly and robustly precluded. Philip Pettit thus argues 
for the desirability (and indeed necessity) of collective 
rationality with his example of a political party: “the party 
cannot tolerate collective inconsistency, because that 
would make it a laughing-stock among its followers and 
in the electorate at large; it could no longer claim to be 
seriously committed to its alleged purpose” (Pettit, 2003, 
p. 178). Examples for the importance of (this sense of) col- 
lective rationality include cases where groups are trusted 
(be it by their members or by outsiders, e.g., as testifiers); 
in such cases, it does indeed seem plausible to assume that 
groups need to achieve, at the collective level, the sort of 
robust coherence that characterizes rational individual 
agency. For a group testimony that p to function as such, 
there has to be trust in collective rationality. One might 
object, however, that any such argument presupposes 
what is claimed to be shown, that is, that group agency 
matters (for an instructive discussion, cf. Roth, 2014). 

In an Arrovian vein, List and Pettit (2011) show that 
collective rationality is incompatible with the counter- 
parts of Arrow’s conditions for preference aggregation (cf. 
List, 2011; cf. chapter 10.1 by Dietrich & Spiekermann, 
this handbook). The upshot is—very roughly—this: if the 
participants can have a rational view on everything that 
is at stake, and if all the individuals’ views are to count 
for the collective judgment, there is no simple aggrega- 
tion rule that robustly ensures collective rationality. This 
severely limits the prospects for the sort of coherence 
that seems necessary to take a group seriously as a ratio- 
nal agent. (Similar to Arrow’s theorem, one might object 
that the simple majority rule works rather well in most 
cases of collective decision making and that the problem 
with the simple majority rule arises only in the special 
case of interdependent decisions; however, a plausible 
counterargument is that trusting a group requires some 
modal robustness and that a group that cannot ensure 
collective rationality in the complicated case cannot be 
trusted in simple cases.) 

The debate on social choice has thus again come to 
disagree with Buchanan’s “individualistic principle”: 
it is argued that collective rationality is desirable after 
all. However, the result so far is only grist to Buchanan’s 
mill in that it appears that under the stated conditions, 
no application of a simple aggregation rule will robustly 
ensure collective rationality, even if all the individual 
inputs fed into the process are rational. 

One way to interpret this result is that collective 
rationality cannot be conceived of in terms of an aggre- 
gation function. Collective rationality is not a matter of 
organizational design. Collective rationality cannot be 
achieved reliably by feeding rational individual inputs 
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into a mechanism that yields a collective attitude. 
Admittedly, some groups do need such organizational 
mechanisms—think of modern democracy. (And it is 
no coincidence that modern social choice theory origi- 
nates from investigations of voting procedures in mass 
society.) But what the impossibility results have shown 
might just be that thinking of collective rationality on 
this paradigm is problematic or even wrong-headed. Per- 
haps such organizational measures as voting procedures 
are attempts to emulate collective rationality under unfa- 
vorable circumstances. Smaller groups typically resort to 
votes only if in the process of deliberation dissent proves 
to be persistent; in many other cases, groups work out a 
joint view in the same way they might jointly prepare 
a meal—not by mechanically assembling whatever the 
participants happen to bring along but by working out 
a joint plan and a division of tasks. Thinking about col- 
lective rationality on the model of voting mechanisms 
might be mistaking the emulation for the original. 

Indeed, in the last chapter of their book on Group Agency 
(2011, chapter 9), List and Pettit sketch a “behavioral” 
alternative to the “organizational” route out of the aggre- 
gation problem. Roughly speaking, the view is that for 
groups to get their act together, they have to self-identify 
with the team in a way that is “immediate” in the sense 
that it leaves no “identification gap.” An identification 
gap opens up whenever members think of their group in 
third-personal terms rather than adopting the first-person- 
plural perspective (e.g., by my thinking of my professional 
group in terms of “the philosophy department” rather 
than in terms of us). If, from my perspective, the consis- 
tency problem is “the philosophy department’s,” it is not 
my problem. Thus conceived, the group is another agent, 
over and above the heads of its members, and collective 
irrationality seems to be a problem only if I am, for some 
additional reason, bothered by it. From a first-person- 
plural perspective, however, inconsistency is our problem, 
and it involves me in a way that does not run through any 
extra reason for me to be bothered by it. 


4. Collective Intentionality 


Even though these two conceptions of collective ratio- 
nality differ in the sort of practical problems they 
concern as well as in the notions of rationality and col- 
lectivity involved in each of the conceptions, they point 
toward one and the same issue: the two solutions to 
the two different problems of collective rationality con- 
verge on the topic of thinking and acting as a team. The 
sort of reasoning behind collective rationality appears 
to be collective in a way in which the participants are 
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self-identified as members that is not in need of further 
individual motivational or evidential support. But how 
could individuals ever be “blindly” identified with their 
teams in such a way that their group’s interests and con- 
sistency requirements should motivate (and perhaps 
even justify) their contribution without an extra indi- 
vidual motivation? 

One way to address this issue is via the first-person- 
plural perspective. In recent philosophical research, this 
has mostly been investigated under the label collective 
intentionality, that is, “the power of minds to be jointly 
directed at objects, matters of fact, states of affairs, goals, 
or values” (Schweikard & Schmid, 2013). Although phi- 
losophers disagree over how to analyze collective inten- 
tional states, the first-person-plural perspective figures 
prominently in this debate. From the members’ points 
of view, their team’s attitude (or lack thereof, or incon- 
sistency thereof) is not just some other agent’s busi- 
ness, which concerns them only as external constraints 
on their own actions. Rather, it is theirs, collectively. 
Put in first-personal terms, that is why it concerns me, 
as a member, in a way that is different from a third- 
party interest or consistency problem, and much closer 
to the way in which my own interests and consistency 
problems concern me. The way I am committed by my 
interests, and bothered by my inconsistencies, is via the 
particular way I know them, which is self-knowledge, 
self-consciousness, or self-awareness. One way to cash 
out the capacity for joint thought and action further 
is thus in terms of plural (prereflective) self-awareness 
(Schmid, 2014, 2018). The way in which team reasoners 
and self-identified members know, or feel, to be reason- 
ing together self-commits to a shared goal in the same 
way in which an individual preference commits one to 
an individual goal, although the cases are not identical. 
In the individual case, an attitude of the form “x real- 
izes best what I value, but why should I do it?” makes 
little rational sense (although it may occur in the case 
of weakness of will): judging that an alternative is best 
is to be committed to choosing it in the same way as 
judging that something is true is to believe it; there is, 
under normal circumstances, no room for questions 
concerning whether or not a proposition should be 
believed after its truth has been settled. In the collec- 
tive case, however, judgment and rational choice seem 
to come apart, to some degree. To judge, as a team, that 
x is best for us leaves some room open for rational ques- 
tioning whether or not J should contribute (one might 
need some assurance from others before deciding to pull 
one’s weight). It should be noted, however, that this plu- 
ral case is still very different from a third-personal case. 
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“Our” judgment commits me to a different degree than 
my own, but in contrast to an outsider’s judgment, it does 
commit me. If I decide to act against what we, together, 
judge to be best, I certainly owe an explanation, while the 
converse is not true: our judgment provides a pro tanto 
reason for my contribution (cf. Gilbert, 2013). After all, 
the view in question is ours, and that implicates me—it is 
first-personal, although not in the singular form, but in 
the plural. Thus, our judgment self-commits me (I need no 
reason to be committed by a judgment besides knowing it 
to be ours) in the same general way as my own judgment 
self-commits me just in virtue of my self-awareness of it. 

This also bears on the question of consistency, and 
again, the analogies and disanalogies to the singular case 
are instructive. Our individual minds contain inconsis- 
tent attitudes aplenty, but becoming aware of them (in 
the right first-personal way) is being committed to sort- 
ing them out. I can perhaps have incompossible inten- 
tions, but as soon as I become first-personally aware of 
them, they lose their status as practical commitments, 
and the issue concerning what to do is reopened. Plu- 
ral self-awareness works in the same way: it is in vir- 
tue of our first-person-plural knowledge of attitudes as 
ours that we cannot be unbothered by their potential 
inconsistencies but have to make up our (plural) mind 
about what to think and what to do. In this way, plu- 
ral (prereflective) self-awareness constitutes a collective 
consistency requirement in the same way as singular 
(prereflective) self-awareness constitutes an individual 
consistency requirement. In this way, collective self- 
awareness makes it not only desirable but possible, too, 
to be effective and consistent as the teams we are. 
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10.3 Rationality in Communication 


Georg Meggle 


Summary 


Communicative actions are actions aiming at being under- 
stood. This is the common ground of many different the- 
ories of communication. Explicating what is entailed in 
this smallest common denominator is the first and fore- 
most task of a general theory of communicative actions. 
This chapter covers three different kinds of communica- 
tive theory: an individualistic theory of communication 
following ideas of Paul Grice (section 1) and two col- 
lectivist theories of communication (section 2), namely, 
speech act theory, initiated by John L. Austin and John 
Searle, and the theory of communicative actions as 
developed by Jiirgen Habermas. Finally, in section 3, we 
examine various aspects of rationality involved in and 
presupposed by these different approaches. 


1. The Individualistic Approach 


This chapter is not on communication in general, but 
only about communicative actions. A general theory of 
such communicative actions is thus part of a general 
theory of actions. Its main question is, What is the dif- 
ferentia specifica of communicative actions? And regard- 
ing this handbook, is the rationality of communicative 
actions really a form of rationality sui generis (as vigor- 
ously argued by Habermas)? 

Communicative actions are actions aiming at being under- 
stood, and understanding an action is just knowledge of its 
meaning. This is the common ground of all sorts of com- 
munication theories. Thus, explicating what is entailed 
by this common ground is the foremost task of the the- 
ory sought after. However, these explications divide over 
the readings of “action,” “action understanding,” and 
“action meaning” they are starting from. 

Regarding actions, one has to distinguish between 
action types (e.g., the action type singing) and action 
tokens (e.g., my singing now). Theories whose basic 
concepts are primarily defined for action tokens, as per- 
formed by their respective agents or subjects at particular 


times, follow the so-called research program of method- 
ological individualism. Theories following the alterna- 
tive program of methodological collectivism start from 
concepts basically related to action types. Notice that 
methodological differences need not entail ontological 
differences. It is this methodological difference that sets 
the stage for the following overview.' 


1.1. Basic Concepts 
Communicative actions are actions aiming at being under- 
stood. This is enough for them to qualify as instrumental 
actions, that is, as actions by means of which the agent 
tries to achieve her respective aim(s). 

Instrumental or intentional actions can be defined 
quite easily by combining the three basic (qualitative) 
elements of any (individualistic) action theory: 


D(X,f) := X does or performs (at time ft) the action fj 


B(X,A) := X believes (strongly)—i.e., is convinced— 


(at ft) that A; 
W(X, A) := X wants or desires (at t) that A. 


These are the three basic notions on which everything 
else will build. First the notion of intentional action: 


Definition 1: By doing f, X intends to achieve aim A iff 
(if and only if) X does f, X wants that A, and X believes 
that A will come about if X does f; in symbols: 


(x, fA) = D(X, f) A W(X, A) A BX, DX, f) <— A). 


Here, <> stands for material equivalence.” Here and 
in the following, it should be strengthened to a sub- 
junctive biconditional. However, we better neglect this 
complication. 

Such an action is successful iff X’s belief, her expecta- 
tion of success, is correct, that is, iff D(X,f) <© A indeed 
holds. Sometimes our aims have a certain hierarchy. If 
in X’s view, her aim A; is merely a means to achieve aim 
Aj, aim A; is her primary aim regarding A2. Therefore, the 
primary aim, if it exists, is the aim that is primary regard- 
ing all other aims. 
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What, then, does it mean to understand an action? It 
is common to say that we understand an action of X iff 
we know the reasons for which it was done. These rea- 
sons should be spelled out more generally with reference 
to X’s beliefs and desires, or preferences and probabilities 
(see chapter 2.1 by Broome and chapter 8.2 by Peterson, 
both in this handbook). However, since our focus is on 
intentions (definition 1), we may conveniently restrict 
our considerations to the following special case: we 
understand an action iff we know the intention with 
which it was performed. 

Since intentions are subjective (in the sense that having 
an intention is being convinced of having it), this agrees 
with Weber (1913/1988, pp. 429, 432), who says that we 
understand an action iff we know its subjective meaning. 
Here, the subjective meaning of an action is nothing but 
the meaning or function this action has for the agent. And 
this we know when we know the corresponding intention 
with which the action was performed. 

Let us make our notion of understanding a bit more 
explicit. For the sake of simplicity, we may equate 
knowledge with true belief (conviction): “X knows (at f) 
that A” = K(X,A) :=B(X,A) a A, although we are aware 
that since Plato, this equation is taken not to be correct. 
Then our notion comes to 


Definition 2: Y (fully) understands X’s doing f iff Y knows 
all the aims which X intends to achieve by doing f; 
symbolically: U(Y, D(X, f)) := K(Y, (x, f,A*)), where A* 
represents the sum of all aims X intends by doing f- 


The upshot is that the understanding of actions and 
the rationality of actions are two sides of the same coin. 
Every rational action is understandable, and according to 
definition 2, only rational actions are understandable— 
trivially so: reasons, and intentions in particular, can be 
(fully) known to us only if they exist. 


1.2 Communicative Actions 

Let us apply now these basic notions to communica- 
tive actions. Typically, such actions are utterances (of 
sentences). However, they may also be gestures, facial 
expressions, flying a flag, and so on. Almost anything 
may serve communication. We shall focus here, pars pro 
toto, on one general form of communication attempts, 
namely, so-called informatives, where the primary com- 
municative aim of speaker S is to get the hearer H to 
believe some proposition p. 

So, S’s doing f is an (informative) communicative 
attempt addressed at H with the content p, only if by 
doing f, S intends to make H believe (at t’ > f) that p; in 
symbols: CA(S,H,f,p) > \(S,f,B'(H, p)).? 
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Communicative aims may also be achieved in a non- 
communicative way. The crucial question hence is, What 
distinguishes a communicative action f from a merely 
(noncommunicative) instrumental action with the same 
aim? It is obviously the special communicative way by 
means of which S expects to achieve her primary com- 
municative aim, namely, through H’s very recognition of 
her communication attempt f. That is, S believes that H 
comes to believe p (at t’) iff H recognizes (knows at ¢’) that 
S’s doing f is such a communicative attempt. Formally: 
CA(S,H, fp) > B(S, B'(H, p) <> K'(H, CAS, H, f,p))). 

When we take the two conditions on communicative 
attempts just stated to be jointly sufficient, we arrive at 
the following adequacy condition: 


AC1: CA(S,H,f, p) = 1(S,f, B'(A, p)) « B(S, B'(H, p) <= 


K'(, CAG, H, f, p))). 


Note that this is not a definition of CA; as such, it 
would be circular. Hence, it only serves as a condition 
that must be derivable from any acceptable definition 
of CA. 

The most important conclusion from AC1 is the fol- 
lowing reflexivity condition: 


RC: CA(S,H,f,p) > US, f,K'(, CAG, 4, f,p))). 


In terms of definition 2, RC states the crucial principle 
that communication always aims at being understood. 
It follows from AC1, if one accepts the principle that 
believed means and consequences of intended aims are 
intended as well. 

The next step is to realize that this reflexivity con- 
dition in turn spawns further consequences and entails 
the unlimited transparency or openness of communica- 
tive intentions. That is, the communicative attempt has 
the primary intention Ip = I(S,f,B'(H,p)) that H should 
believe p. Via RC, this entails that I, = I(S,f, K’(H, Io)) also 
belongs to the intentions of the communicative attempt. 
Then, so does I = I(S, f, K'(H,)), and so on. In general, if 
I, is a communicative intention, so is I... = I(S,f, K'(A, ,)) 
for any n=O. In this way, the reflexivity condition RC 
extends to 


RC*: CA(S,H,f,p) > (Sf, KEI"), 


where J* represents the conjunction of the hierarchy just 
developed; it is, we might say, an unlimitedly transpar- 
ent communicative intention. 

However, we still don’t know what an adequate expli- 
cation of CA looks like. Reviewing the various answers 
proposed would be a lengthy procedure (sketched in 
Meggle, 1981). Let us cut the long story short and just 
look at its beginning and its end. It starts with Grice’s 
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basic model (in Grice, 1957, p. 384), which says the fol- 
lowing in our present terms: 


CA(S,H, f, p) := \(S, f, B'(G, p)) » BS, BEL, p) < K'(A, ho). 
And we have just developed the end of the story: 


Definition 3: CA(S,H,f, p) = 
I(S, f, B'(,, p)) « B(S, B'(A, p) <> K'(,I*)). 


The difference between J, and J* seems small—merely 
an asterisk. Yet this asterisk harbors what we have called 
“unlimited transparency” or “openness.” Note that defi- 
nition 3 indeed entails the adequacy condition AC1, 
since, according to this definition, knowledge of the com- 
municative attempt CA(S,H,f,p) amounts to knowledge 
of the unlimited intention J*. 

In definition 2, we explicated understanding an 
action in general. This applies as well to communicative 
actions and results in 


Definition 4: Person Y (who may be the hearer H 
himself) understands S’s doing fas a communicative 
attempt addressed at H with the content p iff Y knows 
that S undertakes such a communicative attempt. In 
symbols: U(Y, CA(S,H, f, p)) := K(Y, CA(S, H, f, p)). 


The definition takes this simple form because know- 
ing of the communicative attempt CA(S,H,f,p) entails, 
via the strong reflexivity condition RC*, knowing of the 
speaker's unlimited intention I*, with which she performs 
this attempt. 

Let’s still attend to the hearer’s side. Suppose that 
you understand S’s action as a communicative attempt 
addressed at you with the content p. When would you 
actually correspond to S’s involved expectation of suc- 
cess, that is, actually believe p? Normally only if you take 
both conditions, 


Sincerity: CA(S,H,f,p) > B(S,p), and 
Correctness: B(S,p) — p, 


to be satisfied, that is, that S does not tell a lie and does 
not err. I refer to these conditions as “communicative 
normality conditions.” They may or may not be actu- 
ally satisfied in a particular given case. On the individ- 
ual level on which we have moved so far, a theory of 
communicative actions may be silent on such normality 
conditions. However, we will see that they play a crucial 
role on the social level. 


1.3 Intersubjective Meaning 

If the individualistic approach is to be successful, the 
social level needs to be construed on its basis. Explain- 
ing how this might work in principle is the goal of this 
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section. A first noteworthy point is an ontological one: 
whereas subjective meanings (i.e., intentions) are those 
of a communicative action token as performed by a sub- 
ject S (at a time £), intersubjective meanings are attached 
to action types relative to a collective or a population P 
and, moreover, relative to a certain type of situation 2. 
The meaning of a certain gesture or utterance may vary 
from group to group and even from situation to situa- 
tion within one group. Again, we better speak of action 
types generally and not specifically of utterance types. 
Now, what might intersubjective meanings be? How 
to explicate M(P, %, F, p): that action type F has for P in = 
the intersubjective meaning that p? It seems clear that such 
meanings must be common knowledge in the group P. 
We follow the usual definition and say that p is common 
knowledge in P—symbolically: K*(P, p)—iff everybody in P 
knows that p (K(S, p) for all S in P), everybody in P knows 
that everybody in P knows that p, everybody knows this 
in turn, and so on (see also chapter 9.2 by Perea on com- 
mon belief, this handbook). Thus, intersubjective mean- 
ings should satisfy the following adequacy condition: 


AC2: M(P,3,F,p) > K*(,M(,2,F,p))- 


On the basis of our above treatment of the individual 
level, it is not difficult to arrive at an adequate explica- 
tion of M. We might start with the proposal of a mere 
communicative regularity (where the index 0 at Mo signals 
that there is still a caveat). 


Definition 5: Mo(P,2,F,p) := for each speaker S in P and 
each action token f of type F performed in a situ- 
ation of type =, there is a hearer H in P such that 
doing f is a communicative attempt with the con- 
tent p: D(S, f) > CA(S,H,f,p).* 


We might call Mo “regular meaning,” since it states 
only a regularity in communication tokens. The caveat 
is that Mo does not yet fulfill our adequacy condition 
AC2. But this is easily filled by requiring that regular 
meaning be common knowledge: 


Definition 6: M(P,%,F, p) := K*(P, Mo(P, 2, F,p)). 


It may be easily shown that this definition satisfies 
AC2; if the regular meanings are common knowledge, 
so are the intersubjective meanings (for all details, see 
Meggle, 2003, pp. 244-251). 

Definitions 5 and 6 are the essential steps from (sub- 
jective) meanings on the action-token level to (intersub- 
jective) meanings on the action-type level. Notice that 
the latter meanings are defined in terms of—but not iden- 
tified with—the former ones. (This difference is mostly 
disregarded or misinterpreted in the criticisms of the 
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individualist approach. This misinterpretation was unfor- 
tunately invited by Bennett [1973], one of the champions 
of the Gricean doctrine. Already the title of this paper, 
“The Meaning-Nominalist Strategy,” suggested that con- 
ventional meaning is just a special case of speaker’s mean- 
ing, CA(S, H, f, p). This is false, of course.) 

An essential step in our account is still missing. On 
the individual level, a speaker S makes a communica- 
tive attempt CA(S, H, f, p) with two sorts of expectations, 
namely, the expectation that the hearer understands her 
attempt, B(S, D(S, f) = K'(H, CA(S, H, f, p))), and that her 
attempt is indeed successful, that is, B(S, K'(H, CA(S, H, f, p)) 
— B'(H,p)). It is not obvious how these expectations 
transfer to the social level. On which reasons can they be 
based in order to make them well founded on the action- 
type level? 

Concerning the expectation of understanding, the 
answer is just M(P,2,F,p) itself and its entailed common 
knowledge; each speaker expects her audience to under- 
stand her. However, concerning the expectation of success, 
there is obviously a rationality gap left open by definition 
6. Up to now, there is nothing in M(P,2,F,p) on which 
this expectation could regularly keep relying. But without 
such a reliable base, the communicative regularity of 
M(P, 2, F,p) would be miraculous and could not be stable. 

The missing link adding stability to regularity is con- 
ventionality, which is based not only on common knowl- 
edge but also on common interests. For the locus classicus 
of a game-theoretical modeling of how such interests or 
preferences turn regularities into conventions, see Lewis 
(1969); see also chapter 9.1 by Albert and Kliemt (this 
handbook). Lewis’s explication of signaling conventions 
is an explication of conventional meaning as a special 
case of M(P,2,F,p) (see Skyrms, 2010, or the reconstruc- 
tions in von Kutschera, 1983, and Meggle, 2010). 

The way conventionality closes the rationality gap 
behind M(P,2,F,p) is best explained by an idealized 
story, in which 2 = * is a perfect communication situation, 
which means (i) it is a common interest of S and H in =* 
that B'(A, p) <= p; (ii) it is common knowledge among 
S and H that K(S, p) or K(S, =p), but neither K(H, p) nor 
K(H, =p); and (iii) it is also common knowledge among 
S and H that S and H will follow a joint strategy con- 
sisting of S’s strategy D(S,f) <p and of H's strategy 
B'(H, p) + D(S, f). 

Now, suppose that S$ in P makes the communicative 
attempt CA(S, H,f, p) toward H in P, in such a perfect situ- 
ation 2*, that fis of type F and that indeed the intersub- 
jective meaning of F in =* is p, that is, M(P,2,F,p). Then 
the normality conditions of sincerity and correctness 
(section 1.2) do hold. This is entailed by clause (iii) about 
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common knowledge of the joint strategy, while clauses 
(i) and (ii) explain why S even makes the effort of a com- 
municative attempt. In this way, the best reasons for the 
expectation of success at the CA-token level (i.e., the con- 
ditions of sincerity and correctness) are transformed also 
into the best reasons at the CA-type level. 

Clearly, perfect communication situations are very 
idealized. The actual picture is statistically blurred every- 
where. However, the ideal picture is the guideline for all 
attempts at more realism. Here, the point was only to 
explain the individualistic strategy for construing inter- 
subjective meanings in terms of subjective communica- 
tive actions. 


1.4 Toward the Semantics and Pragmatics 

of Expressions 

So far, the individualistic approach has delivered a mean- 
ing theory for action performances (e.g., whistling, sing- 
ing, or flying a flag). However, these performances must 
be distinguished from their products or results (whistles, 
songs, or the flags flied). Thus, we get the following matrix 
comprising the four different levels: 


Action performance Action product 


Action type (ID) 
Action token (1) 


Product form (II) 
Product token (IV) 


Type 
Token 


We have dealt with I and II. So, the remaining big ques- 
tion is, How do we get from an action-theoretic seman- 
tics for communicative actions to a semantics for action 
products (signs or expressions)? Let’s turn first to level III 
and then to level IV. 

For simple (unstructured) signs (like green or red 
lights, flags, etc.) the solution is simple, too: their mean- 
ings can be identified with the conventional mean- 
ings of their performances. So, if F-doing is the way of 
producing the sign- or utterance-type U, we may equate 
M(P,~, U,p) = M(P,2,F,p). However, this trivial identifi- 
cation in the case of unstructured signs is not available 
when structured signs enter the field. Then the question 
arises as to how the meaning of a complex sign is a func- 
tion of the meanings of its parts. And as every linguist 
will agree, there is no generally accepted answer to this 
question up to now. 

As for structured expressions—linguistic expressions 
being their most important case—a truly systematic action- 
theoretic semantics would have to start from an exclu- 
sively syntactically characterized language (say, predicate 
logic amended by all kinds of indexicals and intensional 
and performative operators) and deliver an interpretation 
of this language, as paradigmatically designed in modern 
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intensional semantics. And it would have to show how 
this interpretation can be systematically based on corre- 
sponding communicative conventions stated at level II 
(a task already solved by Lewis [1969], extended by von 
Kutschera [1983], and embedded into a full account of 
level I by Meggle [2010]). Eventually, the entire feat would 
have to be carried over to natural languages. Thereby, 
however, we would get deeply involved in linguistics. 

Let us finally turn to level IV, accounting for the com- 
municative meanings of concrete utterances u of type 
U. Since a semantics for utterance types U (level III) is 
already presupposed here, level IV is the field of tradi- 
tional pragmatics. Thus, the main question on this level 
is how best to systematize the various literal and nonlit- 
eral ways of doing things with signs and words in con- 
crete situations (in a communicative form). 

Although pragmatics is the most prolific field in 
communication theory and has shown already tremen- 
dous progress with regards to formalizations in the past 
decades (see, e.g., Horn & Ward, 2004), a systematic prag- 
matics based on, and formulated in terms of, a general 
communication theory is still missing. For quite obvious 
reasons: conventional sign meanings in the group P are 
relative to various (idealized) situation-types 2. Thereby, 
literal sign-usages in a specific situation o of type = are 
associated with mutual communicative expectations 
firmly grounded in the conventions behind M(?,X, F, p); 
how this works was indicated at the end of section 1.3 
for those perfect situations. However, what the mutual 
communicative expectations actually are in the situa- 
tion ois an entirely open question, because o has many 
other features beyond being of type =. Each feature is 
potentially relevant; this might involve everything com- 
monly known from human social life. 

Well, relevant for what exactly? (See Wilson & Sper- 
ber [2012] and chapter 4.3 by Merin, this handbook.) 
This crucial question is best answered by returning to 
the beginning of our individualistic account, where we 
tried to capture communicative understanding. 

We started with the question of what communicative 
understanding means in the case of a totally unspecified 
action token, and now we have arrived at the same ques- 
tion for concrete utterances of action types with mean- 
ings already specified in relation to particular populations, 
situations, languages, and other structured instruments of 
human interaction (argumentation rules and other cul- 
tural rules and habits in the fields of knowledge, sciences, 
religion, jurisdiction, arts, etc.). However, all these kinds 
of factors are relevant only insofar as they contribute to 
the simple question explicitly left open at the beginning 
(but required to get a good answer at this final level): how 
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can speaker S addressing hearer H in this specific situa- 
tion o reasonably believe her expectation of understand- 
ing and of success to be fulfilled? Well, all these factors 
are relevant insofar as they are part and parcel of the best 
rational explanation of S’s communicative attempt. 

There are, from ancient rhetorics (see chapter 5.6 by 
Woods, this handbook) over speech act theory, Haber- 
mas’s universal pragmatics (both to be discussed in a 
moment), Grice’s theory of implicatures (Grice, 1989), 
and the just-mentioned relevance theory up to mod- 
ern linguistic pragmatics, many proposals to structure 
this extremely broad field of possibly relevant factors by 
various sorts of conversational rules or maxims (see, e.g., 
Korta & Perry, 2006/2020). However, all of this is beyond 
the scope of this chapter. 


2. Collectivistic Approaches 


2.1 Speech Act Theory 

Whereas an individualist theory of communicative 
actions starts with actions, as well as their meaning 
and understanding, in the subjective sense, speech act 
theory—which is perhaps the leading collectivist version 
following the meaning-as-use approach of Wittgenstein 
(1953)—proceeds from actions already endowed with an 
intersubjective meaning. In terms of the matrix above, 
speech act theory starts at level II and not at level I. 

Accordingly, the basic question of speech act theory 
is not how to define communicative actions in general 
but how to account for the different forms of these acts 
as specified in the following open list of so-called illocu- 
tionary roles: commanding, asking, questioning, thank- 
ing, threatening, warning, recommending, dissuading, 
declaring, excusing, promising, and so on. This has been 
the central concern of speech act theory as founded by 
Austin (1962) and developed by Searle (1969) (see also 
Vanderveken, 1990-1991). Unfortunately, though, no 
explicit definition has been presented going beyond 
such an open list of illocutionary roles or acts having 
these roles. So, on the one hand, speech act theory was 
very fruitful in many applications in philosophy and lin- 
guistics, but on the other hand, it was also in need of 
conceptual explication from the start. 

In fact, it is not quite correct to say that speech act 
theory starts at level II. Although it focused on illocu- 
tionary roles and acts, it presupposed that these acts 
are utterances of signs already having a conventional 
linguistic meaning.’ Thus, it was exclusively interested 
in linguistic communication. In other words, it started 
rather at a combination of levels II and III, where the lat- 
ter was assumed not to be in need of further clarification. 
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This assumption has initially barred speech act the- 
ory from developing a general pragmatic semantics that 
explicates meaning also for structured (linguistic) action 
products. The first to notice and try to overcome this bar 
was Alston (1964; and in more detail in Alston, 2000); he 
was followed by von Savigny (1983) and in quite differ- 
ent ways by Brandom (1994). 

However, let me rather dwell on how speech act 
theory construes the relation between levels I and II. 
It is here where its differences with the individualistic 
approach in section 1 become most conspicuous. A cen- 
tral distinction maintained in all versions of speech act 
theory is that between illocutionary and perlocutionary 
acts, the latter ones being acts of actually bringing about 
something by intentionally doing or uttering some- 
thing, with or without the help of conventions. And 
this fits exactly our explication of instrumental actions 
as presented in definition 1. In short: intentional perlo- 
cutionary acts are instrumental acts. 

Now, the central claim of speech act theory associated 
with this distinction is the irreducibility thesis: illocution- 
ary acts are not definable in terms of perlocutionary 
acts, and illocutionary forces are not definable in terms 
of perlocutionary effects; both are sui generis. (The locus 
classicus here is Austin, 1962, chapter 9.) 

Is the irreducibility thesis correct? This depends on 
what kind of illocutionary acts we are talking about. Fol- 
lowing earlier suggestions of Strawson (1964) and Schiffer 
(1972), it is mainly due to Bach and Harnish (1979) that 
we should distinguish between two kinds of illocutionary 
acts: they are either communicative acts or institutional 
acts. Paradigm cases of the latter would be the (proce- 
dural) act types of getting married, baptizing, or betting 
at an auction—the very cases Austin had started from. 

Now, by performing such an institutional act, one may 
indeed try to achieve various things (say, enrichment, 
in-group acceptance, or eternal redemption). The point, 
though, is that no special perlocutionary aspect, no spe- 
cial instrumental intention, is conceptually required for 
an act to count as institutional. Thus, for this kind of 
illocutionary acts, the irreducibility thesis holds trivi- 
ally. However, this does not entail that it holds also for 
the other sort of illocutionary acts, namely, the commu- 
nicative ones. On the contrary, as shown in section 1, 
the thesis is simply false at least for the informatives 
discussed there. There we found that some (communi- 
cative) illocutionary intentions are a special sort of per- 
locutionary (instrumentalist) intentions. Hence, at least 
as far as the informatives, an embracive class of com- 
municative illocutionary acts, are concerned, speech act 
theory could be individualistically reconstructed on the 
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basis of the explications given in section 1, as shown by 
Schiffer (1972) and Meggle (1981). 

The most basic issue between an individualist and 
a collectivist approach to meaning revolves around 
what kind of explication of M(P,2z,F,p) one is willing to 
accept. Whereas the individualist would accept a reduc- 
tionist explication as given in section 1, this very sort 
of reductionism is denied by the collectivist. He would 
agree to the adequacy condition AC2 for intersubjec- 
tive meaning stated in section 1.3. However, he cannot 
accept the individualistic explication of M as presented 
in definitions 5 and 6 (section 1.3). He is committed to 
give M and its corresponding notion of understanding 
a different reading, one that refers to the relevant rules 
defining what is or counts as a correct performance of the 
respective action type F in P and ¥. His notion of under- 
standing is simply this: we understand a communicative 
action iff we know its rules.° 


2.2 Habermas’s Theory of Communicative Actions 
Communicative actions, as this term is used in Habermas 
(1981), are of a very different sort from all those con- 
sidered so far. Habermas conceives of them as collective 
actions, performed by a collective P and not by a single 
individual, as assumed in the individualistic approach 
(section 1) and in speech act theory (section 2.1). If such 
individual actions of members of P are to add up to a 
communicative action in Habermas’s sense, they must 
integrate into some kind of common practice in which 
the members of P engage with a certain shared attitude. 
It is this shared attitude that characterizes Habermasian 
communicative action for a collective P in a situation 
= with respect to some state of affairs p. This attitude is 
best described as embracing three connected parts: (i) a 
cognitive, (ii) a voluntative, and (iii) a praxiological part. 

Ad (i): it is common knowledge in P; (i.e., the subgroup 
of P involved in 2) that it is (maybe only tentatively) an 
open question for P; whether p or not-p, where p refers to 
a factual (empirical) or expressive (referring to one’s state 
of mind) or normative state of affairs. Ad (ii): it is a com- 
mon interest in Ps; to engage jointly and open-mindedly 
in trying to reach mutual understanding—in the sense of 
mutual agreement (consensus) in Ps—about whether p or 
not-p is the case. And ad (iii): the procedures to achieve 
this common goal are those of giving and taking reasons 
in consideration of p. They are governed by the essen- 
tial characteristics of our everyday speech acts, and their 
rules agree with, and could be explicated in terms of, the 
argumentation rules of rational theoretical or practical 
discourse (see also chapter 5.5 by Hahn & Collins and 
chapter 5.6 by Woods, both in this handbook). 


Rationality in Communication 


As suggested by part (iii), this characterization of Haber- 
masian communicative actions utilizes, as he intends, 
some strongly idealized aspects of speech act theory. Fol- 
lowing Biihler’s (1934) tripartition of the organum model 
of speech events (with its three poles of sender, recep- 
tor, and things and, respectively, its relations of expres- 
sion, appeal, and representation), Habermas’s speech 
act model distinguishes between three worlds (reflected 
in part (i) above), with which every speech act is corre- 
lated: the objective, the social, and the subjective world, 
which correspond to the distinction between culture, 
society, and personality. According to Habermas’s theory, 
these three dimensions show up in three corresponding 
dimensions of communicative rationality. Communica- 
tion aims at the discursive redemption of validity claims 
in ways structurally and universally embedded in our 
everyday speech act practice. This aim is at the center of 
Habermas’s attempts at a unified concept of rationality. 
These attempts were strongly assisted by his friend K.-O. 
Apel. Apel’s work on transcendental pragmatics’ incorpo- 
rated Peirce’s pragmatic conception of truth as consensus 
reached in the long run by an indefinitely extended com- 
munity of discourse and thus set the stage for the speech- 
act-theoretic backings of Habermas’s theory. 

As should be clear from this very condensed summary, 
Habermas’s project of spelling out the consequences 
of the above basic characterization of communicative 
actions in his sense, and applying them to the various 
possible social and political fields, is much richer than a 
specific theory such as the one sketched in section 1. It 
covers a whole research program, which we cannot even 
sketch here. Moreover, one has to keep in mind how 
Habermas develops and presents his program, namely, 
in his typical multiperspective style by means of system- 
atic and historical reconstructions and by combining 
philosophical accounts with empirical approaches from 
psychology and the social and political sciences. Com- 
monly, such research programs can only be meant as 
work in progress from the outset. 

Already this very brief outline shows that switching 
from the notion of communicative actions as developed 
in section 1 to Habermas’s notion implies a radical para- 
digm shift. The shift may be required, since Habermas 
intended his theory to be a starting point for a new foun- 
dation of social theory, particularly of a theory of mod- 
ern societies and their transnational extensions. Since 
this paradigm shift has often been disregarded (maybe 
even by Habermas himself—see the last paragraph of 
this section), let us at least notice some of its aspects. 

For this purpose, we should keep in mind that we 
distinguished above between (a) instrumental acts, (b) 
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communicative attempts in general, and (c) conven- 
tional and linguistic communicative attempts in partic- 
ular and, in section 2.1, between (d) communicative and 
(e) institutional illocutionary acts. Thus, if one would 
want to systematically reconstruct Habermas’s theory, 
one would have to ask how it relates to these distinc- 
tions or, more generally, how Habermas’s collective 
communicative actions are correlated with the various 
communicative and illocutionary acts on the individual 
or on the group level. Let me only make a few sugges- 
tions in this direction. 

For instance, Habermas’s theory is committed to the 
irreducibility thesis (section 2.1). Hence, one would have 
to know how its speech-act-theoretic foundations look like 
without this thesis, in order to fairly compare his theory 
with the individualistic approach at the general level (b). 

Next, according to Habermas, communication is 
essentially a consensus-oriented activity. This famous 
claim can be subscribed to by a Gricean individualist, 
too. However, this orientation toward consensus would 
have to be spelled out quite differently for the two sides. 
For ease of comparison, let us restrict the group P to $ 
and H. Then, according to the above sketch of Haber- 
mas’s approach, S and H try to reach common knowledge 
about the truth of either p or not-p. In the individualistic 
approach, by contrast, a communicative attempt of the 
form CA(S,H,f,p) only entails that S intends by doing f 
that she will bring about common knowledge between 
her and H that CA(S,H,f, p). So, on both sides, there is nec- 
essarily some openness involved. In Habermas’s account, 
however, it is a joint open-mindedness related to some 
unknown factual or normative state of affairs p of com- 
mon interest, while in the individualistic approach, this 
openness is essentially restricted to the communicative 
state of affairs CA(S, H, f, p) itself. 

Moreover, for Habermas, communicative actions are 
opposed to strategic actions, trivially so, because he defines 
strategic actions with reference to their missing openness. 
Therefore, Habermas seems to imply that strategic actions 
are also the opposite of communicative actions in the 
individualistic sense. However, there is no such opposi- 
tion. As explained in section 1, communicative actions in 
the latter sense are instrumental actions; they need not 
even be sincere, although they have to be so sufficiently 
often on the conventional level. 

Finally, what about Habermas’s central tenet that com- 
municative actions are actions sui generis? That is, how 
do Habermasian communicative actions relate to the 
various actions types (a)-(e)? There is an easy first answer 
confirming this tenet: since collective actions in general 
transcend the contribution of all individual actions, they 
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are by definition sui generis. And so are, hence, Haber- 
mas’s communicative actions. However, nothing follows 
from this with respect to whether or not even these com- 
municative actions may be ultimately explicated in terms 
of more basic individualistic concepts. 


3. Rationality Assumptions—An Overview 


3.1 Rationality in the Individualistic Approach 

Let us look back on our route and see what kinds of ratio- 
nality assumptions we have used at the various stages. And 
let us first take up the individualistic approach (section 1). 
There we first came across considerations of rationality 
when we explained a general concept of understanding 
an action as knowing its reasons. These reasons lie in the 
agent’s preferences and beliefs, the yardstick by which 
her action’s rationality is measured. This is the concept of 
action rationality (as dealt with in chapter 2.1 by Broome 
and chapter 2.2 by Wedgwood and the entire section 8 in 
this handbook). 

Our explanation assumed that these reasons in turn 
correspond to certain standards of rationality that they 
have to satisfy in order to qualify as reasons in the first 
place. Which standards? They certainly include the 
requirement that both, the beliefs and the desires in 
question, are consistent. One might impose the require- 
ment of deductive closure: 


B(X,A) a B(X,A — B) — B(X,B). 


One might extend this to the desires. In any case, 
such principles concern the rationality of our attitudes. 
This is not the place to discuss them in detail, but see, 
for example, chapter 5.1 by van Ditmarsch, chapter 8.1 
by Griine-Yanoff, and chapter 11.1 by Horty and Roy (all 
in this handbook). (My own compass has been Lenzen, 
1980.) Of course, they would have to include principles 
stating rationality conditions among the various atti- 
tudes, such as the following bridge principle: 


W(X,A) a B(X,A — B) > W(X,B).8 


Such principles, which mediate among various atti- 
tudes held by a person at one time, need to be com- 
plemented by principles that mediate between these 
attitudes at different times. Again, this is not the place 
to expand on these issues (see chapter 5.2 by Rott; chap- 
ter 5.3 by Kern-Isberner, Skovgaard-Olsen, & Spohn; 
chapter 5.4 by Gazzo Castafieda & Knauff; and section 5 
of chapter 8.1 by Grtine-Yanoff, all in this handbook). 
Finally, the fact that the agent has sufficient reasons for 
a given action, and that those reasons in turn satisfy the 
relevant rationality requirements, does not mean she will 
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actually do it. She will only do it if she herself is rational 
in the given situation. So, in a nutshell, we must distin- 
guish three aspects of rationality already within general 
action: action rationality, rationality of an action’s rea- 
sons, and personal rationality. 

Owing to the thoroughly subjective nature of action 
rationality, the question of whether the agent’s attitudes 
themselves are well founded in turn remains open, espe- 
cially whether her action (as Weber, 1913/1988, puts it) 
is rational not only in a subjective means—end sense but 
also in an objective-correctness sense (by being based on 
true beliefs). The latter is required only for a successful 
intentional action. 

What do communicative actions add to all this? To wit, 
nothing! Except certain more special expectations on the 
part of the speaker. The speaker expects the hearer to under- 
stand her communication attempt and thus to assume 
of her all three forms of rationality. Moreover, as a result 
of the reflexivity of communication, she expects that the 
hearer also realizes that the speaker expects him to make 
these rationality assumptions of her, and so forth. Hence, 
communicative rationality is nothing but a special case of 
general action rationality plus its openness intended by the 
speaker.’ The soundness of the reasons of a communicative 
action pertains only to its success. However, within the 
framework of a general theory, one can initially avoid say- 
ing anything about whether the expectations of under- 
standing and success have to be sound themselves. 

The picture changes as soon as the types of communi- 
cative actions receive an intersubjective meaning. If 5 and 
H belong to the relevant population, the expectation of 
understanding is supported by the relevant meaningful 
situations. Hence, a communicative attempt is not only 
subjectively rational regarding its aim of being understood 
but also objectively rational in such a situation. Moreover, 
the expectation of communicative success must be con- 
firmed with sufficient frequency. Otherwise, the relevant 
communicative regularity would sooner or later collapse. 
Communication by means of signs or expressions with a 
regular, conventional, or linguistic meaning differs from 
communication without this backup precisely by the regu- 
lar confirmation of the communicative aims. The struc- 
ture of this confirmation can be explicated—following the 
ideas of Lewis (1969)—by applying game theory as a collec- 
tive version of rational decision theory (see chapter 9.1 by 
Albert & Kliemt, chapter 9.2 by Perea, chapter 9.3 by Alex- 
ander, chapter 10.2 by Schmid, and chapter 10.4 by Raub, 
all in this handbook). However, note that “collective” is to 
be understood here only in sensu diviso; the agents are, as in 
decision making in general, individual persons, members 
of a collective, but not the collective itself. 


Rationality in Communication 


With this subjective and intersubjective backing of 
the expectations of understanding, further communi- 
cative spaces open up, especially for “communication 
between the lines,” as studied, for example, in Grice’s 
theory of conversational implicatures, which is based 
on his “cooperative principle.” This is the fundamental 
principle of his theory but a derived principle of coop- 
erative rationality (see Grice, 1989). 


3.2 Rationality in Collectivistic Approaches 

Let us turn to how communicative rationality may be 
conceived within speech act theory. If speech act theory 
were robbed of the irreducibility thesis (section 2.1), 
then it would coincide with the individualistic theory. 
Or so we have argued at least with respect to informa- 
tives. Hence, the relevant rationality assumptions would 
coincide as well. 

But what about the collectivist version of speech act 
theory including the irreducibility thesis? In this case, 
the central concept of understanding, with which our 
search for rationality assumptions involved in commu- 
nication started, does not refer to the intention with 
which the relevant communicative action was per- 
formed. It rather relates to the rules with reference to 
which a correct performance of the relevant action type 
is defined. Now, just as one might say that rationality 
is what makes an action correct in a certain sense, one 
might say that it is the constitutive rules for illocutions 
that make them correct according to speech act theory. 
Compare, for example, if you want to play chess (where 
it would be redundant to add “correctly”), you have to 
stick to the chess rules. But this would not give us a ratio- 
nal explanation why you would like to play chess right 
now or why you prefer to make this move rather than 
that. So, even in speech act theory, one would need to 
refer to the above notions of action rationality. 

Finally, as far as Habermas’s theory of communica- 
tion is concerned, I do not know of any overall ana- 
lytic reconstruction, although there are some promising 
beginnings (see Steinhoff [2009] and Heath [2003], 
the former in a more destructive, the latter in a more 
constructive way). In particular, I do not know of any 
serious attempts to connect Habermas’s collectivistic 
account with the field of collective intentionality richly 
developed in the past 25 years (see the other chapters of 
section 10 in this handbook). That is regrettable, since 
communicative actions in Habermas’s sense may be 
expected to play a central role within this field. With- 
out doubt, it would be very promising to develop Haber- 
mas’s program further within the framework of theories 
of collective intentionality. 
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Notes 


1. For more details, see Meggle (2010). 


2. The point of the equivalence is this: requiring only the weaker 
belief B(X, D(X, f) — A) would allow many actions to be believed 
to be sufficient for the aim A. However, in the theory of commu- 
nicative actions, we would like to be able to explain why exactly 
action f and not any other action sufficient for A is performed. 
And this explanation requires the stronger belief as stated. 


3. For directives, the corresponding primary communicative 
aim would be that H performs some action a. The logical struc- 
ture of directives and informatives is the same: just substitute 
the aim D'(H, a) for the aim B’(H, p). 


4. This definition, indeed the entire section, is replete with 
universal quantifiers. Of course, this is only an ideal type. Actu- 
ally, all these quantifiers should be statistically weakened to 
high percentages, as has been suggested by Lewis (1975). 


5. Cf. Austin’s explanation of what he called a “locutionary 
act” in Austin (1962, pp. 92-93): it is an utterance with a lin- 
guistic meaning “in the sense of Frege.” 


6. For the different sorts of rules constitutive for communi- 
cative meanings in language games versus rules of chess, see 
Meggle (1985) and, in the same spirit, Kemmerling (1992). 


7. See Apel (1976, 1981) and Kuhlmann (1992). 


8. For instance, Kant in Groundwork of the Metaphysics of Morals 
(1785/1903, B 44-45) states, “He who wants the purpose (given 
that the reason has a decisive impact on his actions) also wants 
the means necessary for the purpose.” The quote suggests that 
the material implication within the belief operator needs to be 
strengthened to a subjunctive conditional or a strict implication. 


9. That is, the speaker expects common knowledge of rational- 
ity, as studied in epistemic game theory (see chapter 9.2 by 
Perea, this handbook). 
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10.4 Rational Choice Theory in the Social Sciences 


Werner Raub 


Summary 


This chapter is on applications of rational choice theory 
in the social sciences rather than on rational choice the- 
ory as such, with a focus on social sciences other than 
economics and, more specifically, on sociology. These 
applications employ variants of rational choice theory 
as outlined in the chapters on individual rationality and 
decision making as well as those on game theory in this 
handbook (chapters 9.1-9.4). The applications aim at 
theories in the sense of more or less formalized models 
yielding explanations and testable predictions. Assump- 
tions on rational choice are important elements of such 
models but not the only ones. The chapter sketches 
selected features of applications, including methodologi- 
cal individualism and model building relating micro- and 
macro-levels of analysis, as well as strategic interdepen- 
dence and unintended consequences as core ingredients 
of applications. Some remarks on empirical research and 
suggestions for further reading conclude the chapter. 


1. Applications of Rational Choice Theory in Various 
Social Science Disciplines 


Applications of rational choice theory in the social sci- 
ences are not at all restricted to economics. They mean- 
while abound, too, in disciplines such as political science 
and sociology. Economists have applied rational choice 
theory outside typical domains of economics. Becker’s 
(1976) “economic approach to human behavior” is a 
prime example. Arrow (1963), Buchanan and Tullock 
(1962), Downs (1957), and Olson (1971) have been influ- 
ential in establishing rational choice approaches in politi- 
cal science. An early textbook by political scientists has 
been Riker and Ordeshook (1973). Ostrom’s study on 
governing the commons (1990 and subsequent contribu- 
tions) is an example of work by a political scientist who 
won the Nobel Prize in Economic Sciences. In sociology, 
rational choice theory and its applications are often asso- 
ciated with Coleman (1990). He built on earlier work (see 


Coleman, 1994), including behavioral sociology (Homans, 
1961) and social exchange theory (Blau, 1964; Homans, 
1958) as well as contributions, starting in the 1970s, by 
European sociologists such as Boudon, Esser, Lindenberg, 
Opp, and Wippler, often labeled “structural individual- 
ism” or “explanatory sociology” (Wippler & Lindenberg, 
1987). Meanwhile, applications of rational choice theory 
are an established approach in sociology (Wittek, Snijders, 
& Nee, 2013). 


2. RCT, Methodological Individualism, Micro—Macro 
Models, and an Example 


We use “RCT” for, roughly, “theories in the social sci- 
ences that include, but are not restricted to, assumptions 
from rational choice theory” such as standard axioms for 
rational behavior under certainty, risk, and uncertainty 
in the sense of Harsanyi (1977) and the corresponding 
implications on utility maximization, or game-theoretic 
equilibrium assumptions (see chapter 8.2 by Peterson 
and chapter 9.1 by Albert & Kliemt, both in this hand- 
book).' RCT employs methodological individualism 
(Udehn, 2001) as an approach to theory construction, 
accounting not only for the behavior of individual 
actors but also, and particularly, for phenomena and 
processes at the level of social systems” made up by those 
actors. Therefore, RCT includes two levels, namely, the 
level of actors, often referred to as the “micro-level,” and 
the system level, often referred to as the “macro-level.” 
Moreover, and particularly, RCT focuses on how both 
levels are related. 


2.1. Coleman's Diagram 
Coleman (1986, 1990) has suggested a popular diagram 
(figure 10.4.1) summarizing the “logic” of micro—macro 
models in the social sciences and of RCT explanations 
(for related earlier work, see Raub & Voss, 2017). 

With “macro,” Coleman refers to systems, such as a 
family, a city, a business firm, a school, or a market. “Micro” 
refers to the actors making up the system. The macro-level 
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Coleman’s micro—macro diagram. 


thus relates to collective phenomena described by con- 
cepts referring to properties of social systems, for exam- 
ple, the size of a group. In terms of the number of actors 
involved, there are large as well as small social systems 
like dyads, triads, or small groups. The micro-level relates 
to properties of actors, for example, their preferences, 
beliefs, and behavior. Typically, the actors making up the 
system are individuals. However, depending on the appli- 
cation and simplifying assumptions deemed acceptable, 
the micro-level may include “corporate actors” (Cole- 
man, 1990). For example, in theories of intraorganiza- 
tional processes, a firm may constitute the system level, 
with individual employees and managers as actors on the 
micro-level. Conversely, in theories of interorganizational 
networks, a set of firms may constitute the system level, 
with individual firms as corporate actors on the micro- 
level. Or consider a market model with individual con- 
sumers and firms as producers on the micro-level, firms 
again being conceived as “unitary actors.” 

In Coleman’s diagram, nodes A and D represent propo- 
sitions describing macro-conditions and macro-outcomes, 
respectively. Arrow 4 represents propositions about empiri- 
cal regularities at the macro-level, say, a statistical asso- 
ciation between macro-conditions and macro-outcomes. 
Macro-outcomes (node D) as well as empirical regularities 
(arrow 4) represent explananda at the macro-level. Node 
B represents propositions describing micro-conditions. 
These propositions refer to “independent variables” in 
assumptions about regularities of actors’ behavior, in our 
case, rational choice theory. Arrow 1 represents “bridge 
assumptions” (Wippler & Lindenberg, 1987) on how social 
conditions affect these variables. For example, social condi- 
tions such as networks or institutions, but also prices, can 
be conceived as opportunities or, conversely, constraints 
affecting the feasible alternatives between which actors can 
choose. Social conditions likewise shape incentives associ- 
ated with feasible alternatives and actors’ information. 
Node C represents micro-outcomes and the explanandum 
on the micro-level, namely, descriptions of actors’ behav- 
ior. Rational choice assumptions are represented by arrow 


2. Thus, arrow 2 represents a micro-theory. Finally, arrow 3 
represents assumptions on how actors’ behavior generates 
macro-outcomes. We use “transformation rules” (Wippler 
& Lindenberg, 1987) as a label for such assumptions on 
micro-to-macro relations. It is evident from the diagram 
that the explanandum at the micro-level, descriptions of 
individual behavior, follows from an explanans compris- 
ing assumptions on rational behavior and “initial condi- 
tions” on, for example, actors’ preferences and _ beliefs 
(arrow 2, node B), macro-conditions (node A), and bridge 
assumptions (arrow 1). The explananda at the macro-level, 
namely, descriptions of macro-outcomes (node D) and 
macro-regularities (arrow 4), follow from an explanans 
comprising rational choice assumptions (node B, arrow 2), 
macro-conditions (node A), bridge assumptions (arrow 1), 
and transformation rules (arrow 3). 


2.2. An Example: Collective Good Production 

and the Volunteer’s Dilemma 

As an illustration, consider the production of collective 
goods and the empirical regularity at the macro-level 
that group size is often negatively related to the produc- 
tion of collective goods (Olson, 1971). The core feature 
of a collective good is that actors who did not contrib- 
ute to its production cannot be excluded from its con- 
sumption. Hence, when costs of individual contribution 
are high compared to marginal effects of the contribu- 
tion on individual benefits from the good, actors face 
incentives not to contribute—the free-rider problem. 
Assume furthermore that there are no “selective incen- 
tives” and, hence, no additional individual benefits that 
depend on individual contributions to the production 
of the collective good. Then, Olson argued, collective 
good production will be negatively related to group 
size. However, the relationship between group size and 
collective good production should not be considered a 
simple macro-law. Rather, the relationship depends on 
a number of specific conditions such as the absence of 
selective incentives, the production function for the col- 
lective good, and others. Diekmann’s (1985) Volunteer’s 
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Dilemma (VOD) is a formal model of a set of conditions 
that imply the group size effect. 

The well-known bystander intervention and diffu- 
sion of responsibility problem is a social situation for 
which VOD is a reasonable model: actors witness an 
accident or a crime. Everybody would feel relieved if 
at least one actor would help the victim by calling the 
police. However, providing help is costly, and each actor 
might be inclined to abstain from helping, hoping that 
someone else will help. VOD captures these features in a 
noncooperative game with N actors. Actors have binary 
choices. They decide simultaneously and independently 
whether or not to contribute to the collective good, that 
is, to provide help. The good is costly and will be pro- 
vided if at least one actor—a “volunteer”—decides to 
contribute. Contributions by more than one actor are 
feasible, and then each actor pays the full costs of provid- 
ing the good, but contributions of more than one actor 
do not further improve the utility level of any actor. A 
core feature of VOD is that the costs K of contributing to 
the collective good are smaller than the gains U from the 
good. The matrix in table 10.4.1 summarizes the normal 
form of the game, with rows representing an actor’s strate- 
gies, namely, to contribute (CONTR) or not to contribute 
(DON’T), columns indicating the number of other actors 
who contribute, and cells representing an actor’s payoff as 
a function of his’ own strategy and the number of other 
actors who contribute. 

In terms of Coleman’s diagram (figure 10.4.2), both 
being a noncooperative game and group size N are macro- 
conditions (node A). The macro-outcome of interest 
(node D) is the probability that the collective good will be 
provided. Arrow 4 now represents the relation between 
group size and this macro-level probability. Node B 
represents the micro-conditions (a) that each actor can 
choose between CONTR and DON’T; (b) actors’ infor- 
mation, namely, that actors, when choosing, are not 
aware of the other actors’ choices;* and (c) actors’ pref- 
erences as represented by their payoffs. Note that the 


A: VOD with group size N 
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Table 10.4.1 
The Volunteer’s Dilemma (U>K> 0; N= 2) 
Number of other actors choosing CONTR 
0 1 2 N-1 
CONTR U-K U-K U-K U-K 
DON’T 0 U U U 


normal form of the game includes bridge assumptions 
(arrow 1) on macro—micro transitions. Namely, the nor- 
mal form specifies how an actor’s payoff depends on his 
own choices as well as those of all other actors—that is, 
the normal form specifies the structure of the actors’ 
interdependence. 

Game-theoretic rationality assumptions such as the 
assumption of equilibrium behavior are micro-level 
assumptions (arrow 2). In an equilibrium, each actor’s 
strategy maximizes his own payoffs, given the strategies of 
the other actors. VOD has N equilibria in pure strategies. 
These are strategy combinations with exactly one volun- 
teer choosing CONTR with probability 1, while all other 
actors choose DON’T with probability 1. In each of these 
equilibria, the collective good is provided with certainty. 
However, the equilibria involve a bargaining problem: each 
actor prefers the equilibria with another actor volunteer- 
ing to the equilibrium where he himself is the volunteer. 
Moreover, while the game is symmetric, the N equilibria 
in pure strategies require that actors do not choose the 
same strategies. It is a rather natural assumption that 
rational actors, in a symmetric game, play a symmetric 
equilibrium, that is, choose the same strategies. It can be 
shown that VOD has a unique symmetric equilibrium in 
mixed strategies. In this equilibrium, each actor chooses 


1 
CONTR with probability p* = 1-(F)" <1. Under 


game-theoretic rationality assumptions, the symmetric 
equilibrium in mixed strategies is a plausible candidate 
for the “solution” of VOD (for various issues related to 


D: Probability P* of collective 
good production 


>@ 


B: Individual incentive 
to CONTR 


Figure 10.4.2 
Micro-macro diagram for the Volunteer’s Dilemma. 


C: Individual probability p* 
of CONTR 
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assuming a mixed strategy equilibrium as the solution 
of a game, see Camerer, 2003; Osborne & Rubinstein, 
1994). Thus, p* is represented by node C. Note that p* is 
a decreasing function of N. Therefore, for VOD and under 
the assumption of game-theoretic rationality, a testable 
implication on how group size affects micro-outcomes fol- 
lows: the individual probability to contribute to the col- 
lective good declines with increasing group size. 

The individual probability p has to be distinguished 
from the macro-outcome: the probability P of collec- 
tive good provision. Note that the normal form of VOD 
includes a transformation rule (arrow 3). Namely, the 
normal form specifies how the probability of collec- 
tive good provision depends on each actor’s individual 
behavior. More specifically, the normal form implies 
that the collective good is provided if and only if at 
least one actor chooses CONTR. In VOD, group size 
affects collective good provision through two different 
mechanisms. First, there is a positive effect of increas- 
ing group size, since it is sufficient that one single actor 
contributes and since all actors contribute with positive 
probability in the symmetric mixed equilibrium. After 
all, with increasing group size, the number of actors 
increases who may decide to contribute. Second, and 
conversely, there is a negative effect of increasing group 
size, since each actor’s individual probability p* to con- 
tribute decreases with increasing N. What is the total 
effect? For the symmetric mixed equilibrium, the prob- 
ability that there is at least one volunteer and that the 
collective good will therefore be provided can be shown 


tobe p* =1- (ey . In Coleman’s diagram, P* is repre- 
sented by node D. Obviously, P* < 1, so that production 
of the collective good may fail. Moreover, P* is a decreas- 
ing function of N: the negative effect of increasing group 
size on collective good production outweighs its posi- 
tive effect. Hence, assuming game-theoretic rationality, 
a testable implication, represented by arrow 4, follows 
for how group size affects macro-outcomes in VOD: the 
probability that the collective good is provided decreases 
with increasing N. 


3. RCT and Model Building 


The VOD example illustrates typical features of RCT. 
First, VOD is a formal theoretical model. RCT has affin- 
ity with more or less formalized model building. Cole- 
man’s diagram is highly stylized. It looks simple by 
leaving implicit that full-fledged micro—macro models 
typically comprise a sizable number of possibly complex 
assumptions, including assumptions other than those of 
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rational choice theory as such. The VOD example shows 
this. The core task in analyzing such models is to iden- 
tify implications of these assumptions, more specifically 
implications on how micro-level behavior depends on 
macro- and micro-conditions as well as implications 
on how macro-level outcomes and macro-level regu- 
larities depend on macro- and micro-level assumptions, 
including assumptions linking the levels. Deriving such 
implications is often nontrivial. Therefore, formal model 
building, or at least the formalization of certain assump- 
tions, can be helpful, if not a prerequisite, for deriving 
implications. In the VOD example, purely verbal rea- 
soning suffices to see that increasing group size has two 
opposite effects on the probability that the collective 
good will be provided. However, formalization seems 
necessary to derive the total effect. An early version of 
this argument in sociology is in Coleman’s (1964) dis- 
cussion of “synthetic theories.” These theories aim to 
identify the consequences of a set of assumptions and 
more precisely to identify consequences on the macro- 
level for a set of assumptions on the micro-level: “it is 
characteristic of many of these theories that they begin 
with postulates on the individual level and end with 
deductions on the group level” (Coleman, 1964, p. 41). 
Related arguments include that formalization allows for 
checking whether implications do follow from assump- 
tions and for identifying assumptions that are implicitly 
used in merely verbal accounts (for thorough early dis- 
cussions, see Hummell, 1973; Ziegler, 1972; for a recent 
outline, see Tirole, 2017). A major advantage of RCT is 
that it lends itself to formalization of core assumptions 
and provides analytical power by allowing for the con- 
struction of tractable models, specifically models linking 
macro- and micro-levels. 

Second, RCT, like model building in general, typically 
involves a trade-off: simplifying assumptions preserve 
tractability and analytical power for the derivation of 
implications at the cost of (some) assumptions being less 
realistic empirically, while more complex and realistic 
assumptions make it more difficult to derive implica- 
tions. Therefore, RCT typically starts with a model that 
is as simple as possible, making simplifying assumptions 
explicit. Subsequently, when simplifying assumptions 
turn out to be problematic, one introduces more com- 
plex assumptions in a stepwise fashion. This is appro- 
priate when implications are highly dependent on such 
assumptions rather than being robust, or when impli- 
cations fare badly in the light of empirical evidence. 
The procedure is known as the “method of decreasing 
abstraction” (Lindenberg, 1992). For example, the VOD 
employs the simplifying assumption of a symmetric 
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game. More complex versions of VOD allow for individ- 
ual heterogeneity with respect to costs of and gains from 
contributing (Diekmann, 1993). Also, quite a lot of exper- 
imental evidence suggests that the individual probability 
of contributing indeed tends to decline with increasing 
group size, while a decline is not found for the macro- 
level probability that the collective good is provided. 
Tuti¢ (2014) provides a game-theoretic model accounting 
for this pattern by employing a solution concept other 
than the symmetric mixed-strategy equilibrium. 

Third, deriving macro-implications from a micro- 
macro model is a key element of RCT. For example, the 
core issue in the analysis of VOD is not the individ- 
ual probability p to contribute but rather the macro-level 
probability P of collective good production and how 
that macro-level probability depends on group size as a 
macro-condition. Tractability requires simplifications, 
while RCT aims at explanations of macro-outcomes and 
at incorporating macro-conditions in the explanation 
rather than at exclusively explaining individual behav- 
ior as such. Therefore, it seems advisable to keep micro- 
assumptions simple and parsimonious. In the VOD 
example, the analysis becomes feasible through employ- 
ing micro-assumptions on equilibrium behavior. On 
the other hand, it seems reasonable to allow, as much 
as possible, for complexity of macro-assumptions as 
well as assumptions that provide links between macro- 
and micro-levels. In the VOD, therefore, the focus is on 
specifying the structure of interdependence between 
the actors and on how group size, given this interde- 
pendence, affects collective good production. Coleman 
(1987a, 1993) argues furthermore that careful specifica- 
tion of bridge assumptions and transformation rules is 
not only a core requirement for RCT but that explana- 
tions in the social sciences are often deficient precisely 
with respect to bridge assumptions and transformation 
rules. Hence, Coleman assumes that improving bridge 
assumptions and transformation rules will be more ben- 
eficial for theory development than improving micro- 
assumptions. In the VOD example, it is precisely the 
explicit specification of the normal form of the game 
that allows for clearly linking macro- and micro-levels 
since the normal form comprises the relevant bridge 
assumptions and transformation rules. 

Fourth, RCT is consistent with a modest but realis- 
tic view of theory construction in social science, close 
to Merton’s (1968) middle-range theories (Hedstr6m 
& Udehn, 2009): RCT aims at explanations of macro- 
outcomes, including macro-regularities, while the use 
of similar micro-level rational choice assumptions in 
various models addressing different phenomena at the 
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macro-level allows for a common theoretical core and 
coherence of the models as well as cumulative growth 
of knowledge (Diekmann & Voss, 2004). For example, 
VOD can be seen as one element of a family of models of 
social dilemmas (Raub, Buskens, & Corten, 2014). Social 
dilemmas are situations with strategically interdependent 
actors. Actors can choose between cooperation and defec- 
tion. Cooperation by all actors on the micro-level implies 
a Pareto-optimal outcome on the macro-level and is a 
Pareto-improvement compared to defection by all actors: 
at least some actors are better off and nobody is worse 
off when everybody cooperates than when everybody 
defects. However, cooperation of all actors is typically not 
an equilibrium, and at least some actors have individual 
opportunities and incentives to exploit other actors’ coop- 
eration by defecting. Conversely, defection by all actors is 
an equilibrium. The Prisoner’s Dilemma is a classic model 
of a social dilemma. Other well-known models include the 
Trust Game, the Investment Game, and the Public Goods 
Game (see also chapter 9.4 by Dhami & al-Nowaihi, chap- 
ter 10.2 by Schmid, and chapter 10.5 by Nida-Rtmelin, 
Gutwald, & Zuber, all in this handbook). 

For VOD, it follows from properties of equilibria in 
mixed strategies that the symmetric equilibrium is associ- 
ated with expected payoffs U-K for each actor. There is 
another combination of mixed strategies, not being an 
equilibrium, such that each actor contributes with prob- 

1 

ability p"* =1- (a) <1. One easily sees that p** > p*. 
The strategy combination such that each actor contributes 
with probability p** is associated with a Pareto-optimal 
outcome, namely, expected payoffs U- qK for each actor, 
with g <1 and hence U- qK > U-K. Thus, VOD is a social 
dilemma with the mixed strategy of contributing with 
probability p* as defection and the mixed strategy of con- 
tributing with probability p** as cooperation.* 

Models of social dilemmas aim to explain individual 
cooperation or defection at the micro-level and Pareto- 
optimal or Pareto-suboptimal macro-outcomes. The 
models differ by focusing on different macro-conditions 
and their effects, such as group size effects, the effects 
of repeated interactions and networks of relations, the 
effect of various institutions, and so forth. Game-theoretic 
rationality assumptions, or variants of such assumptions, 
ensure a common theoretical core and coherence across 
models of social dilemmas. Moreover, game-theoretic as 
well as other rational choice assumptions similarly ensure 
a common theoretical core not only for different mod- 
els of social dilemmas but also across models for macro- 
phenomena and macro-regularities in a broad range of 
other research domains. 


616 


4. Variants of RCT 


Variants of RCT can be distinguished by considering 
how they differ from two “benchmarks.” One of these 
comprises commonly used assumptions concerning the 
micro-level of actors and their behavior. A second one 
is related to the use of rational choice theory in micro- 
macro models. 


4.1. Variants of RCT: The “Standard Model” 
and Some Alternatives 
Micro-level assumptions of RCT include, but are not 
restricted to, rationality assumptions. In particular, empir- 
ical applications of rational choice theory require addi- 
tional and substantive assumptions. By way of example, 
the rational choice assumption of transitive preferences 
requires that an actor preferring A over B and B over C also 
prefers A over C, but rational choice assumptions do not 
presuppose that A is preferred over B and B over C, nor do 
they comprise assumptions about those properties of A, 
B, and C due to which A is preferred over B and B over C. 
More specifically, rational choice theories do not assume 
at all that actors are exclusively interested in material con- 
sequences for themselves and are purely self-regarding in 
the sense of “utility = own money.” Applications in the 
social sciences do of course require additional and sub- 
stantive micro-level assumptions about actors’ prefer- 
ences alongside the rationality assumptions (see below). 
The same applies to substantive assumptions about 
actors’ beliefs concerning the consequences of choosing 
a certain alternative. However, these assumptions should 
not be confused with the rationality assumptions them- 
selves. It is true that rational choice assumptions are often 
combined with assumptions that actors are purely self- 
regarding and possibly also with assumptions on perfect 
and complete information—so much so that the combi- 
nation of these types of assumptions is sometimes referred 
to as the “standard model” (references in the literature to 
the “homo oeconomicus” typically concern assumptions 
closely related to this standard model). This should not 
distract from the fact that rationality assumptions, on the 
one hand, and assumptions about self-regarding prefer- 
ences or about perfect and complete information, on the 
other, are distinct assumptions of the standard model. 
Some variants of RCT focus on the micro-level and 
offer alternatives to the assumptions of the standard 
model (Dhami, 2016; see chapter 9.4 by Dhami & al- 
Nowaihi, this handbook). Some of these variants modify 
the rational choice assumptions. Experiments like those 
by Kahneman and Tversky (2000) provided evidence on 
regularities of behavior (“anomalies”) that are hard to 
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reconcile with standard rational choice assumptions, 
and they induced the development of alternative micro- 
models of behavior such as prospect theory (Kahneman 
& Tversky, 1979; see chapter 8.3 by Glockner, this hand- 
book), models of bounded rationality (e.g., Rubinstein, 
1998; see chapter 8.5 by Hertwig & Kozyreva, this hand- 
book), dual-process and two-selves models (Kahneman, 
2011), and a variety of models assuming myopic behavior 
and backward-looking learning (Macy & Flache, 2009). 
Meanwhile, Esser’s model of frame selection (Esser & Kro- 
neberg, 2015; Kroneberg, 2014) and Lindenberg’s (2001) 
theory of social rationality are approaches in sociology 
that are inspired by such micro-models and aim at sys- 
tematically developing and employing them in micro- 
macro models of social phenomena (Bruch & Feinberg, 
2017, reviews work in the field of judgment and deci- 
sion making on micro-models that can be employed in 
micro-macro models and are based on other than ratio- 
nal choice assumptions). 

Coleman’s diagram is neutral with respect to the spe- 
cific assumptions on regularities of individual behavior 
represented by arrow 2. Hence, alternatives to standard 
rational choice assumptions such as those mentioned 
may, in principle, likewise serve as micro-level assump- 
tions. Moreover, regularities of behavior hard to rec- 
oncile with standard rational choice assumptions are 
not only found in experiments. Rather, such regulari- 
ties have been likewise documented in field research 
and include micro-level behavior as well as macro-level 
outcomes in core research fields of economics and 
other social sciences (Camerer, 2000). This feature has 
induced attention not only in psychology but also in 
the social sciences for new theories accounting for such 
evidence. 

Other important variants of RCT that deviate from 
standard micro-level assumptions retain rational choice 
assumptions while replacing assumptions such as those 
on self-regarding behavior and likewise employing 
assumptions on information imperfections. Influen- 
tial models combine conventional rationality assump- 
tions with assumptions on inequity aversion or similar 
motives as elements of an actor’s utility function (Coo- 
per & Kagel, 2015; Fehr & Schmidt, 2006). In addition, 
such models assume heterogeneity between actors with 
respect to the strength of such motives and incom- 
plete information of actors about other actors’ utility 
functions. Assumptions on other-regarding preferences 
should be used with care since almost all behavior can 
be “explained” by assuming the “right” preferences and 
adjusting utility functions accordingly (Buskens & Raub, 
2013). Stigler and Becker (1977) even recommended to 
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abstain from explaining variations in behavior across 
actors or over time by assuming variation in utility func- 
tions. Rather, they propose to assume preferences over 
fundamental “commodities” as similar across individuals 
and stable over time, while explaining variations in behav- 
ior through variations in opportunities and constraints 
such as prices, that is, through variations in observable 
macro-conditions. Lindenberg’s theory of “social produc- 
tion functions” is a related approach in sociology (Ormel, 
Lindenberg, Steverink, & Verbrugge, 1999). One would 
prefer first of all parsimony with respect to assumptions 
on other-regarding preferences by adding as few new par- 
ameters as possible in micro-level assumptions. Second, 
assumptions on other-regarding preferences should allow 
for using the same set of assumptions to explain behav- 
ior in a broad range of social situations, thus avoiding 
“tailor-made” adaptations of assumptions for a specific 
explanandum that would render independent testability 
problematic. Third, assumptions on other-regarding pref- 
erences should not only account for well-known regulari- 
ties but should also allow for deriving and testing new 
predictions. From this perspective, it is reassuring that 
models combining rational choice assumptions with 
assumptions on other-regarding preferences, heterogene- 
ity between actors with respect to their preferences, and 
incomplete information about other actors’ preferences 
do indeed yield implications for micro- as well as macro- 
outcomes in a broad range of social situations. The 
models, therefore, preserve tractability and testability, 
despite being less parsimonious than the standard mod- 
els with purely self-regarding preferences. The models 
cover social dilemmas as well as bargaining and distribu- 
tion problems such as the Ultimatum Game and the Dic- 
tator Game, and also include market games (Camerer, 
2003). Moreover, these models show that the same set 
of assumptions implies much cooperative behavior in 
contexts such as social dilemmas but seemingly selfish 
behavior in market contexts. 


4.2. Variants of RCT: Alternatives to the Perfect 
Market Model of Neoclassical Economics 

A second benchmark for variants of RCT is the perfect mar- 
ket model of neoclassical economics. Coleman (1987b) 
regarded that model as—in a sense—a paradigmatic exam- 
ple of an RCT micro—macro model. The characteristics of 
a perfect market (Kreps, 1990) are the relevant macro- 
conditions. Given rational behavior of actors—producers 
and consumers—macro-outcomes of individual exchange 
behavior can be derived, such as the existence of an equi- 
librium with a set of equilibrium prices for the goods, an 
equilibrium distribution of goods, and Pareto-optimality 
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of the equilibrium. The model is exemplary because it is 
nonobvious that the macro-outcomes are implied by the 
assumptions. The model highlights that proving theo- 
rems, and hence showing that certain consequences fol- 
low from assumptions, is a core ingredient of applications 
of rational choice theory in the social sciences. 

Quite a few variants of RCT that focus on the macro- 
level as well as on how micro- and macro-levels are 
related can be seen as alternatives to the perfect mar- 
ket model. Often, these variants retain rational choice 
assumptions and also other assumptions of the stan- 
dard micro-model sketched above, while adapting 
assumptions about macro-conditions. Consequently, 
such variants also focus on bridge assumptions and 
transformation rules other than in the perfect market 
model. For example, Granovetter (1985) argued in his 
programmatic pamphlet that one should replace the 
assumptions of “atomized” actors on perfect markets, 
namely, assumptions of the perfect market model due 
to which strategic interdependence between actors 
can be neglected. Rather, Granovetter maintains, 
one should focus on social structures, that is, macro- 
conditions, that include “embeddedness” of actors. 
“Embeddedness” refers to ongoing relations between 
actors as well as networks of actors. Hence, from a ratio- 
nal choice perspective, Granovetter’s approach focuses 
on macro-conditions that do imply strategic interde- 
pendence. Likewise, his approach focuses on the effects 
of strategic interdependence on individual behavior 
and on macro-effects of that behavior. Granovetter is 
not usually associated with rational choice approaches 
in the social sciences. Nevertheless, his approach is 
strikingly similar to Coleman’s heuristic advice to com- 
bine robust assumptions on rational choice with more 
complex assumptions on social structure and carefully 
designed bridge assumptions and transformation rules. 
Granovetter opposes “psychological revisionism,” char- 
acterizing it as “an attempt to reform economic theory 
by abandoning an absolute assumption of rational 
decision making” (1985, p. 505). Rather, he suggests 
maintaining the rationality assumption: “While the 
assumption of rational action must always be problem- 
atic, it is a good working hypothesis that should not 
easily be abandoned. What looks to the analyst like 
nonrational behavior may be quite sensible when situ- 
ational constraints, especially those of embeddedness, 
are fully appreciated” (1985, p. 506). He argues that 
investments in tracing the effects of embeddedness are 
more promising than investments in the modification 
of the rationality assumption: “My claim is that how- 
ever naive that psychology [of rational choice] may be, 


618 


this is not where the main difficulty lies—it is rather in 
the neglect of social structure” (1985, p. 506). 

Variants of RCT in line with Granovetter’s approach 
and Coleman’s heuristic advice are often models for 
social and economic networks. Other variants focus on 
institutions other than perfect markets, with institutions 
broadly conceived as constraints for human behavior 
that result from human behavior itself and that structure 
the incentives for human behavior (North, 1990). Insti- 
tutions can be of a formal or informal nature, including 
legal infrastructure, on the one hand, and informal con- 
ventions and codes of behavior, on the other. Still other 
RCT variants focus on formal and informal organizations. 
From a sociological perspective, Coleman suggested an 
analogy between RCT alternatives to the perfect market 
model and RCT alternatives to assumptions of the stan- 
dard micro-level model: “Rational choice theory in soci- 
ology occupies a position relative to pure neoclassical 
economics analogous to that of behavioral economics. . . . 
Analogous to the psychological anomalies of behavioral 
economics, which show the systematic deviations from 
rationality that persons exhibit, the social anomalies of 
sociological rational choice theory show the systemic 
deviations from the perfect market assumption of neo- 
classical economies that arise in the linkage between 
micro and macro levels” (Coleman, 1994, p. 167). 

Networks, institutions, and organizations are con- 
ceived as exogenous in some variants of RCT, while 
being endogenous in others. When conceived as exog- 
enous, bridge assumptions can specify how they affect 
individual behavior. For example, as opportunities and 
constraints, they may affect actors’ feasible choices. 
They may also affect actors’ preferences and informa- 
tion. Networks, institutions, and organizations, assumed 
to be exogenous, can likewise affect how actors’ behav- 
iors “combine” to bring about macro-level outcomes and 
transformation rules, with transformation rules specify- 
ing this micro—macro relation. For example, electoral 
institutions may not only affect individual voter behav- 
ior but also affect the macro-outcomes of individual 
voter behavior. This includes that macro-outcomes such 
as the distribution of seats in parliament across political 
parties and, hence, coalition formation can vary consid- 
erably under different electoral institutions, even when 
controlling for individual voter behavior. Other vari- 
ants of RCT focus on networks, institutions, and organ- 
izations as macro-outcomes themselves. Then, networks, 
institutions, and organizations are taken as endogenous, 
and the aim is to explain their emergence and dynamics 
as a result of actors’ rational choice. Often, such mod- 
els will be dynamic in the sense that they account for 
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a social process. The burgeoning literature on strategic 
network formation and the coevolution of networks 
and behavior offers examples of work in this direction 
(Buskens, Corten, & Raub, 2014). From the perspective 
of Coleman’s diagram, the macro-outcome (node D) 
is then simultaneously the “initial” node A of a subse- 
quent micro—macro diagram, and so forth, leading to 
a sequence of “connected” diagrams as in figure 10.4.1 
that account for the development of a social process 
over periods 1, 2,.... 


5. Strategic Interdependence, Game Theory as a Tool, 
and Unintended Consequences as a Topic 


Variants of RCT comprising alternatives to the perfect 
market model show that applications of rational choice 
theory in the social sciences often refer to social systems 
with strategic interdependence between actors. It is there- 
fore not surprising, and illustrated by our VOD example, 
that game-theoretic tools are often employed. After all, 
game theory is the branch of rational choice theory that 
provides concepts, assumptions, and theorems allowing 
to specify how actors behave in interdependent situa- 
tions. Game theory assumes that actors behave as if they 
try to realize their preferences, taking their interdepen- 
dence as well as rational behavior of the other actors into 
account (Harsanyi, 1976). Meanwhile, applications in the 
social sciences employ game theory broadly, including 
not only “classical games” with complete information 
in normal form (Harsanyi, 1977) but also, for example, 
games in extensive form, repeated games, and games with 
incomplete information (Osborne & Rubinstein, 1994). 
With respect to RCT in sociology, interdependencies 
between actors and actors taking their interdependencies 
into account are likewise the core of Weber’s definition of 
social action: “Sociology ... is a science which attempts 
the interpretive understanding of social action in order 
thereby to arrive at a causal explanation of its course and 
effects. ... Action is social in so far as . . . it takes account 
of the behaviour of others and is thereby oriented in its course” 
(Weber, 1947, p. 88, emphasis added). 

Micro- as well as macro-outcomes of an actor’s behav- 
ior typically depend not only on his own choices and 
possibly chance events but also on the behavior of other 
actors. Therefore, outcomes are often unintended conse- 
quences of actors’ behavior: an actor’s intentions need 
not coincide with the outcomes of his behavior. This 
applies, too, under rational behavior of actors (Boudon, 
1977; Elster, 2007; Hayek, 1967; Merton, 1936). Appli- 
cations of rational choice theory in the social sciences, 
certainly applications analyzing situations with strategic 
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interdependence, often focus on outcomes that are unin- 
tended consequences. In the VOD example, rational 
actors maximize their own expected payoffs, given the 
behavior of the other actors, thus producing a Pareto- 
suboptimal macro-outcome, but they are not assumed 
to intentionally produce such an outcome. An important 
distinction is between unintended consequences that are 
beneficial for the actors and unintended consequences 
that are unfavorable. The paradigmatic example for ben- 
eficial unintended consequences is a Pareto-optimal equi- 
librium on a perfect market, driven by selfish actors trying 
to maximizing own benefits, referred to by the “invisible 
hand” metaphor. The paradigmatic example for unin- 
tended consequences that are unfavorable for the actors 
involved are social dilemmas. In these dilemmas, indi- 
vidually rational behavior produces a Pareto-suboptimal 
macro-outcome. The symmetric equilibrium in mixed 
strategies in VOD is an illustration. 


6. RCT and Empirical Research 


Coleman (1986) argued that applying rational choice 
theory in the social sciences might contribute to a bet- 
ter integration of theory and empirical research. Never- 
theless, RCT has for quite some time been criticized for 
weak links with empirical research. Focusing on ratio- 
nal choice approaches in political science, Green and 
Shapiro (1994) argued that much of this work did not 
induce serious empirical research and that empirical 
tests have been poorly conducted (the contributions in 
Friedman, 1996, scrutinize these claims). Meanwhile, the 
situation has changed. For example, Goldthorpe’s (1996) 
arguments for an alliance involving quantitative analy- 
sis of large-scale data sets and rational action theory have 
been influential in sociology (Blossfeld & Prein, 1998). By 
now, quite a lot of work by theorists as well as empirical 
researchers aims at reducing the gap between theoretical 
models based on rational choice assumptions and empiri- 
cal research in sociology. Note that empirical research 
need not and should not be restricted to testing RCT pre- 
dictions. In addition to testing predictions, and certainly 
from the perspective of micro-macro models, empirical 
research is needed to establish macro-level regularities 
(arrow 4 in Coleman’s diagram) that can subsequently 
serve as RCT explananda (for a similar perspective in 
demography, see Billari, 2015; Goldthorpe, 1996, 2016). 

Today, systematic empirical research based on RCT 
is common in many fields, including but not restricted 
to social networks, power and inequality, education 
and social mobility, deviant behavior and criminology, 
religion and secularization, migration and immigrant 
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assimilation, families and households, markets and 
organizations, collective action and collective decision 
making, ecological behavior, and social dilemmas (Abra- 
ham & Voss, 2004; Wittek et al., 2013). While Goldthorpe 
advocated the use of survey research, other research 
designs are meanwhile employed as well (Buskens & 
Raub, 2013; Gachter, 2013). Employing complemen- 
tary research designs, such as lab and field experiments, 
surveys and other observational designs, and vignette 
studies for repeatedly testing the same hypotheses, can 
yield information on the robustness and replicability of 
empirical results (Buskens & Raub, 2013; Jackson & Cox, 
2013). Other important contributions involve the devel- 
opment of statistical models that facilitate better inte- 
gration of theoretical models employing rational choice 
assumptions and empirical research by integrating core 
assumptions on rational choice into the statistical model 
itself (McFadden, 1973; Snijders, 2013). 

Various issues arise related to empirical research 
testing predictions based on RCT (Kroneberg & Kalter, 
2012). These issues include whether such tests should 
exclusively focus on predictions themselves or also on 
empirically assessing assumptions used for deriving 
predictions, such as assumptions on preferences and 
beliefs—and on how to do so. Another issue is how to 
cope with empirical evidence that is hard to reconcile 
with, or an outright refutation of, predictions based on 
RCT. Axioms of rational behavior under certainty, risk, 
and uncertainty a la, for example, Harsanyi (1977) imply 
that a utility function exists and that behavior in accor- 
dance with these axioms can likewise be characterized as 
maximizing the utility function. In a rather strict sense, 
an actor behaving according to the axioms behaves as 
if maximizing a utility function: one need not assume 
that actors consciously calculate (expected) utilities. In 
this sense, rational choice theory is a theory of behavior 
rather than of the underlying mental processes. How- 
ever, behavior may turn out to be more or less systemati- 
cally inconsistent with predictions based on RCT. 

Harsanyi (1976, 1977) argued that although rational 
choice assumptions may yield empirically problematic 
conclusions, it is useful to employ them for establishing 
a “benchmark” so that empirically observed refutations 
become themselves explananda for more refined mod- 
els. Other arguments proceed from the idea that macro- 
rather than micro-level implications of RCT are the 
core issue. From this perspective, it has been suggested 
(Goldthorpe, 1996) that errors in predicting individual 
behavior with RCT will cancel out on the macro-level. 
Becker (1976) has developed the argument that typical 
macro-outcomes of micro—macro models are robust to 
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replacing the rational choice micro-model by alterna- 
tive micro-models. Coleman (1987a, 1990) argues in 
a similar direction and suggests that replacing simple 
rational choice assumptions by more complex micro- 
theories would undermine the tractability of the model 
because it becomes unfeasible to derive implications for 
macro-outcomes at all, certainly so when complex bridge 
assumptions and transformation rules are involved in 
the model. Both the “errors cancel out” argument and 
the argument that macro-outcomes are generically 
robust to varying micro-models of behavior are disputed 
(Raub, Buskens, & Van Assen, 2011). For example, Thaler 
(2018) argues that actors’ deviations from what ratio- 
nal choice assumptions imply do not exhibit “random 
error.” Rather, Kahneman and Tversky’s work, as well as 
subsequent work building on Kahneman and Tversky’s, 
shows that it is predictable how actors’ empirical behav- 
ior deviates from implications of rational choice assump- 
tions. Furthermore, Thaler (2018) presents evidence that 
even macro-outcomes like aggregate market behavior can 
deviate from what one would expect based on rational 
choice assumptions. This would contradict the argument 
of Becker and others that institutions like competitive 
markets with exchange and division of labor operate so 
that nonrational behavior of possibly even many actors 
will be “eliminated” and aggregate outcomes are driven 
by fully rational behavior of possibly only few actors who 
thrive on other actors’ “errors” (Clement, 2002). Another 
controversy with respect to RCT and empirical research 
(Green & Shapiro, 1994; Grofman, 1993) concerns the 
merits and problems of empirical research that focuses 
on qualitative predictions of changes “at the margin” 
using comparative statics rather than on quantitative 
point predictions. 


7. Concluding Remarks and Suggestions 
for Further Reading 


This chapter has been on selected features of con- 
temporary applications of rational choice theory in the 
social sciences. See Macy and Flache (1995), Hechter 
and Kanazawa (1997), and Abraham and Voss (2004) 
for earlier concise reviews of selected applications. Wit- 
tek et al. (2013) is a recent handbook on applications in 
many domains. Ermakoff (2017) is a general discussion 
of RCT in sociology complementary to this chapter. For 
reasons of space, we have neglected a “history of ideas” 
perspective on how contemporary applications relate to, 
say, British moral and social philosophy of the 17th and 
18th centuries (Hobbes, Hume, Smith, Ferguson), the 
Austrian school of economics (Menger, Schumpeter, 
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Hayek), and analytical philosophy of science (e.g., Hem- 
pel, Nagel) and critical rationalism (Popper, Lakatos). 
Udehn (2001) is a source with respect to history of ideas. 
An informative collection of interviews with key schol- 
ars in the social sciences who have been influential in 
promoting and discussing contemporary applications is 
Swedberg (1990). Wittek (2013) is a bibliography with 
pointers to additional references on various issues dis- 
cussed in this chapter. Burgeoning research programs in 
the social sciences are similar to applications of rational 
choice theory in that they focus on micro- as well as 
macro-levels of analysis and try to systematically relate 
these levels as well as assuming variants of incentive- 
driven and goal-directed behavior. These research pro- 
grams include behavioral and experimental game theory 
(Camerer, 2003), approaches employing models of learn- 
ing in games and evolutionary game theory (Fudenberg 
& Levine, 1998; Gintis, 2009a, 2009b; see chapter 9.3 by 
Alexander, this handbook), analytical sociology (Hed- 
strom, 2005; Hedstroém & Bearman, 2009; Manzo, 2014; 
for discussion, see Opp, 2013), and applications of agent- 
based computational modeling (Macy & Flache, 2009). 
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Notes 


1. “RCT” is thus not used as an abbreviation for “rational 
choice theory” but refers to social science theories employing 
rational choice assumptions. 


2. We use the label “social system” broadly, including systems 
of actors typically analyzed in, for example, sociology, political 
science, and economics. 


3. Throughout, we use “he” and “his” to facilitate readability 
and without intending gender bias. 


4. Strictly speaking, we would have to specify the extensive 
form, including the game tree, rather than only the normal 
form of VOD, to make its information structure explicit. 


5. Note, by the way, that it can also be shown that the macro- 
level probability that the collective good will be provided 
increases with increasing group size if each actor contributes 
with individual probability p** (Diekmann, 1986). 
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10.5 Structural Rationality 


Julian Nida-Riimelin, Rebecca Gutwald, and Niina Zuber 


Summary 


This entry focuses on the theory of structural rationality 
(TSR), which goes beyond the traditional individualist 
and instrumental approaches to rationality. The start- 
ing point of TSR is a shortcoming in traditional rational 
choice theory (RCT). In RCT, the rationality of an action 
is evaluated according to its suitability to fulfill the 
agent’s desires at the time of choice, without considering 
future plans or cooperation. TSR is multidimensional, 
because it takes at least three dimensions of rational rea- 
soning into account: decisions regarding future aims and 
plans (diachronic dimension), the plurality of reasons 
for actions (intrapersonal dimension), and interaction 
with others (interpersonal dimension). This account of 
TSR differs from the standard use of “structural rational- 
ity” or rather “structural irrationality” in analytic philos- 
ophy, since the latter focuses merely on formal relations 
between different entities such as desires, actions, and 
intentions. TSR, however, is more radical: it transcends 
the reductive narrowness of instrumental rationality 
without denying its practical impact. 


1. Individualistic and Nonindividualistic Approaches 
to Rationality 


There are two major traditions in the theory of practi- 
cal rationality: the first characterizes practical rationality 
in terms of the acting individual’s properties, capabili- 
ties, and states, whereas the second starts from social 
roles, established practices, and institutions. Within 
both of these traditions, there are numerous variations 
but few commonalities. This conflict of paradigms ulti- 
mately corresponds to a conflict of disciplines: classical 
and neoclassical economic theory are committed to the 
individualistic paradigm, while sociology, cultural stud- 
ies, and social psychology are committed to the second 
paradigm (cf. chapter 10.4 by Raub, this handbook). Of 
course, exceptions do exist: the flourishing of behavioral 
economics (cf. chapter 9.4 by Dhami & al-Nowaihi, this 


handbook) is a reaction to the deficits of the first para- 
digm, trying to also incorporate consideration of the sec- 
ond paradigm, for instance, by including nonrational 
behavior in economic explanation or adherence to social 
norms (Diamond & Vartiainen, 2007; Gigerenzer & Selten, 
2001; Kahneman, Slovic, & Tversky, 1982; Kahneman & 
Tversky, 2000). The theory of structural rationality (TSR) 
is an account that overcomes this conflict of paradigms. It 
takes into consideration the embeddedness of individual 
actions (first paradigm) within the behavioral structures 
of forms of life (second paradigm). This requires a com- 
prehensive redescription of rational agency. 


2. The Paradigmatic Core of the Theory 
of Structural Rationality 


In his Nicomachean Ethics, Aristotle (2000) characterizes 
akrasia (which is usually translated as “weakness of the 
will,” although “lack of self-control” would be more 
accurate) as a singular action that is not embedded in 
an intended or desired structure of the agent’s course of 
action. In this context, it is essential to take into consid- 
eration that Aristotle and the entire classical antiquity as 
well as medieval and even early modern practical philos- 
ophy do not distinguish between a theory of rationality 
and a theory of morality: living the righteous life of the 
dikaios consists in realizing eudaimonia (usually trans- 
lated as “happiness”). This characterization of a virtuous 
life, however, is not individualistic, because it depends 
on the political community (polis) as a whole, in which 
the lives of individuals unfold. Determining what is 
good for a community is the precondition for deter- 
mining what is good for the individual. In other words: 
aspects of structural rationality can already be found in 
classical accounts of rationality, ethics, and politics (cf. 
chapter 12.1 by Fehige & Wessels, this handbook). 

The breakup of the unity of morality and rationality 
that originated in the European era of Enlightenment 
is most radical in the Kantian and the Utilitarian tradi- 
tions. Jeremy Bentham (1780/2007) argues that pursuing 
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one’s own utility for a maximally advantageous pleasure— 
pain balance is the decisive motive for individual human 
action. He deduces the totality of well-being as the guiding 
moral principle. This tension prevails in contemporary 
utilitarianism: what is good is determined on the basis 
of individual well-being or preference fulfillment (Hare, 
1963, 1970, 1984; Harsanyi, 1977a, 1977b; Singer, 1993, 
2016), while moral obligation does not consist in opti- 
mizing individual well-being but in optimizing the sum of 
individual well-being. Rational choice theory (RCT) also 
focuses on individual preferences: actors do not necessat- 
ily optimize their individual well-being but rather strive 
for the fulfillment of their actual, subjective preferences. 

The unity of rationality and morality also disintegrates 
in the Kantian tradition: rationality amounts to optimiz- 
ing one’s own well-being and is guided by pragmatic 
imperatives, as Kant (1785) calls them, while moral action 
is characterized as being exclusively motivated by respect 
for the Moral Law requiring that the individual maxim of 
an action is suitable for being transformed into a univer- 
sal law according to which everybody could act. 

In the Kantian tradition, structural embeddedness is 
realized by adopting only such maxims that are suitable 
for application as universal rules for action. Kantianism, 
however, suffers from being underdetermined, because 
the Moral Law can be met by almost any decision if it 
is described appropriately. This amounts to the critique 
of being formalist that Kantianism has been facing since 
Hegel (1820/2017) and that endures in contemporary 
communitarianism (Taylor, 1975, 1989). Utilitarianism, 
in turn, suffers from collectivism, which is incompatible 
with what John Rawls (1971) called the separateness of 
persons, because it demands maximization of total utility. 
Individual motives and plans of life are not considered. 

It may seem surprising, but ordinary game theory, in 
tying rational decision making to equilibrium points, also 
already incorporates an aspect of TSR and is therefore 
not reducible to decision theory (cf. chapter 9.2 by Perea, 
this handbook). In Harsanyi’s game-theoretic version 
of rule utilitarianism (“Ethical Bayesianism”; Harsanyi, 
1967, 1977c, 1979), the criterion of moral action is the 
joint optimization of a collective value function through 
coordinated individual practice. In this understanding, 
rule utilitarianism is different from act utilitarianism as 
it requires structurally rational action, that is, to act in 
such a way that the combination with the actions of 
others who are also ethically motivated would optimize 
the common value function (the sum of preference ful- 
fillment), whereas act utilitarianism is about optimizing 
overall preference fulfillment with every single act. 
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3. Rational Choice and Structural Rationality 


RCT almost never, or at least never directly, includes the 
embeddedness of pointwise actions in diachronic and 
interpersonal structures. Other agents are considered 
part of the environment that influences the fulfillment 
conditions of personal preferences. However, RCT deals 
with not only situations of risk and uncertainty but also 
interaction. Since this branch of RCT emerged histori- 
cally from the mathematical analysis of games, it is still 
labeled, misleadingly, as “game theory” (Hargreaves-Heap 
& Varoufakis, 2004). Rational choice is determined rela- 
tive to a given value function of consequences and the 
subjective probability function of the circumstances. In 
situations of interaction, these circumstances usually con- 
sist in the decisions of other agents (possibly plus addi- 
tional chance events). Hence, the expansion of the rational 
choice paradigm from decision-theoretic optimization 
(“playing against nature”) to game-theoretic optimization 
appears to be plausible, at least with regard to interactions 
in which the decisions of the players are mutually inde- 
pendent from each other (noncooperative games; cf. chap- 
ter 9.1 by Albert & Kliemt, this handbook). Problems arise, 
however, because every agent must assume that the other 
agents decide rationally, too. The decisions of other players 
ideally follow the same game-theoretic criteria of rational- 
ity. Therefore, game theory renders only those decisions 
rational that are part of an equilibrium point, that is, a com- 
bination of decisions in which none of the agents involved 
has an interest in revoking his decision, as long as all the 
other agents involved stick to their original decisions 
(Nida-Rtimelin, 1994). 

According to game theory, there is only one ratio- 
nal decision for both players, A and B, in the prisoner’s 
dilemma (PD) game described in table 10.5.1: the combi- 
nation of dominant choices D. A vast evidence (Andreoni 
& Miller, 1993; Barreda-Tarrazona, Jaramillo-Gutiérrez, 
Pavan, & Sabater-Grande, 2017; Clark & Sefton, 2001; 
Cooper, Dejong, Forsythe, & Ross, 1996; Kreps, Milgrom, 
Roberts, & Wilson, 1982; Pothos, Perry, Corr, Matthew, 
& Busemeyer, 2001), however, shows that a large num- 
ber of agents chooses C, even in one-shot prisoner’s 


Table 10.5.1 
Prisoner’s dilemma 


Player B 
Cc D 
Player A C 33 1,4 
D 4,1 2,2 
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dilemmas. The orthodox interpretation takes this as an 
indicator that irrational behavior prevails, but this is sim- 
ply not convincing. Decision theory and game theory 
understand themselves as neutral regarding motivation. 
Criteria of rationality concern choosing the best means to 
reach one’s goals (i.e., to fulfill one’s preferences). Imagine 
a person who wants to cooperate in a one-shot prisoner’s 
dilemma. Asked about her motivation for choosing C, she 
might answer, “I choose C because I want that we both 
choose C and because I expect the other person to choose 
C, too.” A general theory of rationality should allow for 
a motivation of this cooperative kind (cf. chapter 10.2 by 
Schmid, this handbook). 

Cooperation is a paradigmatic case of structurally 
rational agency. It is impossible to characterize coop- 
eration adequately if one takes merely outcomes and 
probabilities into consideration (Nida-Rtimelin, 1993, 
chapter 4; 2019). To render cooperation (i.e., the choice 
of dominated C) rational in PD, given the cooperative 
motivations of the players, we cannot dismiss the infor- 
mation of the format itself: the structure of interaction 
is an essential part of the description of strategies as 
cooperative or defective. TSR depends on a comprehen- 
sive description of strategic options. Cooperative action 
embeds the individual choice into intended structures 
of interaction: I do my part, hoping that the other does 
his part to realize a collective action that secures coop- 
eration (i.e., an outcome that is better for each than the 
outcome of individual optimization would be). 

Assuming a form of “collective” or “shared” inten- 
tion represents one way of dealing with the PD chal- 
lenge (Hurley, 2005). Philosophers of collective action 
and intentionality like Raimo Tuomela (2013), Marga- 
ret Gilbert (2013), Michael Bratman (2014), or David 
Regan (1980; see also Nida-Rtiimelin, 2014) have there- 
fore introduced a collective perspective on intention, 
which changes the very outlook on collective-action 
puzzles like the PD. Collective intention amounts to 
changing individual utilities based on a group’s ethos, 
as Tuomela has called it. 

Empirical findings (Axelrod, 2006; Gintis, Bowles, 
Boyd, & Richerson, 2005; Rapoport, Chammah, & Orwant, 
1956; Seabright, 2004; Sethi & Somanathan, 2005) regard- 
ing the behavior of persons in PD situations are best 
explained by interpreting it as structurally rational. The 
individual agents prefer C to D in order to realize (C,C). 
This describes why participants in PD games claim to be 
disappointed if the other person does not choose C and 
also if they chose C. 
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4. Meta-Preferences 


Structurally rational actions challenge traditional RCT, 
particularly the revealed-preference concept and the 
belief—desire model of rationality (Nida-Riimelin, 1991). A 
prominent approach to remedy this problem is the intro- 
duction of meta-preferences, for instance, in the works of 
Amartya Sen and Harry Frankfurt. Sen (1974) argues that 
there are three levels of morality: the lowest level, namely, 
the egoist level, which is captured in PD meta-preferences, 
that is, meta-preferences that accord with first-order pref- 
erences. Second, assurance-game (AG) preferences mani- 
fest themselves in favoring cooperation to defection if 
the other person chooses C, and third, other-regarding 
meta-preferences (OR) translate as choosing C no matter 
how the other person decides. Both of these PD variants 
can be interpreted as disclosing structural rationality: AG 
players are conditional cooperators, because they do not 
want to be exploited by egoists, if they cooperate. OR 
players embed their respective individual choices into the 
preferred structure of interaction (universal cooperation), 
possibly with a moral motivation of the Kantian type. 

Another prominent account of meta-preferences that 
is sometimes thought to adequately describe structural 
aspects of rationality without dismissing the Humean 
belief-desire model of rationality is presented by Frank- 
furt (1971). He proposes a concept of a person as an indi- 
vidual who develops meta-preferences, which means 
that she can distance herself from first-order preferences 
if she has freedom of the will. Second-order volitions are 
meta-preferences regarding first-order action-guiding 
preferences. The idea is that a person is different from a 
“wanton” insofar as she develops second-order volitions 
that frame her first-order action-guiding preferences. 
According to Frankfurt, a “wanton” is not a person in 
the full sense, since she does not care about her will and 
only follows her first-order desires. 

Both types of meta-preferences display structural 
aspects of traditional RCT. Without structural reasoning, 
an individual would be nothing more than a pointwise 
optimizer and would not be perceived as a person who 
persists through time and through different moments 
of interactions (Nida-Rimelin, 2005a, chapter III). We 
establish and maintain structural deliberation by evalu- 
ating our action-guiding preferences. If it turns out that 
our preferences are structurally incoherent, we are criti- 
cal about incoherence and wish we had had different 
first-order action-guiding preferences. This evaluation 
may influence our first-order action-guiding preferences. 
If it does not, there is a conflict of the structural dimen- 
sions described above and classical rational choice. 
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5. The Role of Reasons within the Theory 
of Structural Rationality 


The distinction between the theory of rationality and 
morality collapses if the rationality of an action is char- 
acterized in terms of practical reasons (i.e., of different 
reasons for specific actions). The observation that there 
are good reasons for actions that are not consequential- 
ist can lead to quite different reactions: 


(1) One might assume that these nonconsequentialist 
reasons are merely prima facie reasons that cannot be 
integrated and thus do not constitute genuine good 
reasons (this account could be called “consequential- 
ist”). The problem with this account is that some of 
our most central types of reasons would have to be 
excluded. 


(2) One might confine the range of application of ratio- 
nal choice and exclude moral and other types of good 
reasons for action (this could be called the “narrow 
rationality account”). RCT would then not be an all- 
embracing theory of practical rationality. 

(3) One might give up some of the axioms constitu- 
tive for standard RCT, as Edward McClennen (1990) 
demonstrated in his theory of resolute choice (this 
account could be called “revisionist”). 


(4) One might redesign the conceptual framework, that 
is, by reinterpreting the basic concepts of RC such 
that nonconsequentialist reasons for action can be 
integrated (this is the strategy of TSR; Nida-Rtiimelin, 
2000, 2005b). 


Take the following example: if I have given a prom- 
ise, I have a reason to keep it even if keeping it has less 
than optimal consequences for me or the person I gave 
it to. Speech act theory (Searle, 1969) has analyzed in 
detail the normative elements that constitute the insti- 
tution of giving a promise. This analysis reveals the 
structural character of reasons to act that are connected 
to the speech act of giving a promise: it maintains the 
diachronic consistency of one’s practice in certain types 
of interaction. Keeping a promise generates a duty that 
allows people to relate and forecast the actions of others 
within a larger context. This institution allows for dia- 
chronically and interpersonally structured rationality. 

The same applies to communitarian duties one may 
have because of one’s social role, such as parent, teacher, 
or politician. Communitarian duties constitute these 
roles and sustain an interpersonally and diachronically 
consistent social practice. Individual rights can be inter- 
preted structurally, too. They secure the individual’s 
authorship in social contexts. They enable individuals 
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to act within the boundaries of their individual rights. 
Individual rights thus secure structures of individual 
agency. 

To sum up: the concept of structure is not a kind of 
external appraisal function that is added to the estab- 
lished life-world practice of giving reasons and accepting 
reasons. It is a systematization of this practice. We strive 
for TSR avant la lettre. Without a reference to structures, 
there is no consistent practice, no personhood, and no 
collective rationality. 


6. Is Structural Rationality Instrumental? 


The term “structural rationality” can be found in other 
accounts within the theory of action. The concepts of 
“structural rationality” in this literature are, however, 
merely instrumental. Thus, they focus on individual 
rationality and emphasize the relational and consistent 
properties of rationality. This instrumental understand- 
ing refers to intrapersonal relations between conative 
attitudes; only some approaches include diachronic 
dimensions. The analytic philosophical tradition sticks 
to this account of structural rationality, whereas the TSR 
also highlights interpersonal structural features by con- 
sidering interactions among individuals as a decisive 
part of rationality. In addition, TSR makes substantial 
claims about structural rationality. 

The planning theory of Michael Bratman (2014) is one 
of the few accounts that focus on the diachronic and inter- 
personal aspects of rationality. Bratman thus acknowl- 
edges social rationality norms with regard to intentions 
and policies: friendship, love, dancing together, conver- 
sations, getting married, and so on are all practices and 
forms of action that cannot be explained without refer- 
ence to a form of “modest sociality” (Bratman, 2007). 
His concept of modest sociality provides the explanatory 
basis for our cooperative activities by recourse to concep- 
tual and normative resources that are already in place in 
our understanding of individual agency. This makes Brat- 
man’s explication of cooperation a conceptual extension 
of his theory of individual agency, since the individual 
finds himself always already located within social con- 
texts. Bratman argues that collective intentional agency 
(i.e., cooperative action) is established by shared inten- 
tions. A group sharing an intention goes beyond merely 
intending to do the same thing. They can only act 
together, or in concert, if they adopt a suitable planning 
structure that coordinates their actions and deliberations 
(and if this is common knowledge). If neither of them 
fulfills the relevant (sub)plan, cooperation fails. Bratman 
(2007, p. 8), therefore, applies a quite particular notion 
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of “structures”: intentions are considered plan states that 
take up a certain position in coordinating plans that, in 
turn, establish diachronic and interpersonal structures of 
agency. Intentions have a coordinating role in managing 
conduct and providing continuity and organization over 
time—either in collective action or in individual agency. 

Thomas Scanlon (2007) explicitly uses the term “struc- 
tural irrationality” as an identifier for occurrences of 
irrational behavior. Scanlon is commonly viewed as hav- 
ing coined the term for analytic philosophy of action.' 
In the tradition of analytic philosophy, the term “struc- 
tural rationality” is used as a formal, relational concept 
of consistency, for instance, referring to the consistency 
of the attitudes, intentions, desires, and so on that an 
individual holds simultaneously at one particular point 
in time (Wallace, 2018). Structural rationality does not 
make any substantive claim about the desirability of 
the action’s content; it concentrates on the relational 
requirements that the agent needs to fulfill if she does 
not want to act irrationally. 

To illustrate this relational form of rationality, take 
the following example, introduced by D. J. Langlois 
(2014) as one form of irrationality that he calls “inten- 
tion inconsistency”: your intention is to spend the 
weekend with your family, while you simultaneously 
have the intention to complete your new manuscript by 
Sunday. Also, and this is crucial, you believe that you 
cannot do both. Irrationality arises due to the relational 
link between the two intentions and the belief about 
their incompatibility. Structural irrationality comes into 
existence because it is implausible to state that one con- 
siders such an arrangement as consistent. To accept all 
these propositions simultaneously is not possible since 
they exclude each other; they are incompatible with 
each other. This is, however, not due to their content 
but due to their relations to each other. Neither of them 
is irrational per se (i.e., can be criticized by its content); 
they turn out to be irrational because together they can- 
not form a balanced structure. 

Structural rationality is understood in this manner 
as a nonsubstantial, instrumental, and (at least in some 
form) normative requirement of rationality. Other forms 
of structural irrationality are defined as belief inconsis- 
tency or enkrasia. These types of irrationality have in 
common that the irrationality arises due to the incom- 
patible relationships between singular entities, such as 
intentions, attitudes, and beliefs. Structural irrationality 
is therefore a problem of instrumental rationality since 
we cannot criticize the contents of our desires. We can 
only behave irrationally due to not correctly arranging 
our conative attitudes, including means—end relations. 
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John Broome (1999), too, describes the concept of 
structural irrationality as the failure of some people to 
consistently combine and arrange their attitudes, which 
prevents them from structuring their attitudes ratio- 
nally. Thus, irrationality involves not only forms of 
refraining from action, although one has good reasons 
to act (practical irrationality), or acting without knowl- 
edge or on too little information (theoretical irrationality). 
As with Scanlon’s account, irrationality affects not only 
one or several attitudes but also their arrangement. Such 
an understanding of structural irrationality excludes 
consideration of reasons and only pertains to the rela- 
tions of attitudes. Hence, Scanlon claims that there are 
certain forms of structural requirements that pertain to 
the relations between an agent’s attitudes. This is why 
Scanlon mainly speaks about forms of structural irratio- 
nality: irrationality is structural if one fails to consider 
the adequate relations of attitudes. 

Highlighting the consistent internal relations of ratio- 
nality reduces practical reasoning to instrumental rea- 
soning. Hence, one of the most fundamental questions 
regarding the conceptualization of rationality pertains to 
the rules that determine how attitudes should be com- 
bined and arranged—without referring to their content 
or their objectives.” 


7. Is Structural Rationality Ramsey-Compatible? 


Modern RCT is based on the utility theorem that was 
originally proven by the mathematician and philoso- 
pher Frank P. Ramsey (1978). John von Neumann and 
Oskar Morgenstern (1947) revived this concept of rational 
choice, which was later implemented in different versions 
by Marschak (1946), Luce and Raiffa (1957), and others. 
The utility theorem shows that it is possible to transform 
the qualitative concept of preference into a quantitative 
concept of utility if conditions of consistency, such as 
transitivity of preferences, monotonicity, and continuity 
of preferences regarding lotteries, are met (cf. chapter 8.1 
by Griine-Yanoff and chapter 8.2 by Peterson, both in this 
handbook). In economic application, this theory is mainly 
used to express motivational neutrality of utility functions 
attributed to individual agents (ie., it is irrelevant how 
individuals are motivated). It is viewed as sufficient to 
observe realized preferences. In fact, utility theory requires 
persons only to have consistent preferences (in the sense 
of the axioms); it does not, however, require them to have 
specific motivations. Furthermore, altruistic preferences 
can be described by these axioms and therefore incorpo- 
rated in a cardinal, real-valued utility function.’ The ratio- 
nality of expected utility maximization is deduced from 
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the rationality of individual preferences (via the utility 
theorem), and individual preferences are rational if these 
preferences are consistent (i.e., meet the axioms of the 
utility theorem). 

TSR is not only instrumental. If expected utility maxi- 
mization is the criterion of instrumental rationality, 
then it seems that TSR cannot be Ramsey-compatible. 
This assumption, however, is fallacious, since it assumes 
expected utility maximization, as defined by Ramsey, von 
Neumann, and Morgenstern, to be instrumental. How- 
ever, preferences can reflect motivations related to the 
TSR of an individual as well as comply with Rossian prima 
facie duties (Ross, 2002) or with the Kantian categorical 
imperative or with Aristotelian rules of social practices 
and still be compatible with the axioms of the utility the- 
orem. Therefore, preferences are compatible with the util- 
ity theorem even if they are not instrumental in nature. 
Hence, expected utility maximization is compatible with 
TSR. TSR is therefore compatible with rational choice 
theory. If a structurally rational person makes a decision 
and the relevant probabilities meet the Kolmogorov axi- 
oms and the subjective preferences of the decision maker 
meet the Ramsey conditions, then this person maximizes 
expected utility (or, to use a more adequate term: subjec- 
tive value; cf. chapter 4.1 by Hajek & Staffel, this hand- 
book). Hence, TSR integrates the core idea of RCT: the 
coherence of two types of subjective attitudes—epistemic 
(probabilities) and prohairetic (preferences). 


8. Structural Rationality—Descriptive or Normative? 


There is one salient similarity between traditional RCT 
and TSR: both are normative and descriptive at the same 
time. This feature is most conspicuous in economic theory, 
where economic rationality is used to describe economic 
behavior and at the same time to advise it (Spohn, 2002). 

Looking at the two-sided interpretation of RCT, it 
comes as no surprise that TSR also qualifies as both nor- 
mative and descriptive. It starts with our empirically 
observable behavior and tries to provide an adequate 
description while simultaneously pointing out and criti- 
cizing structurally irrational behavior. It attacks RCT for 
neglecting structures and therefore disregarding impor- 
tant aspects of any coherent practice. We can see how 
structures affect behavior in cases of regret or repentance. 
Structures within theories of rational choice are more 
successful in describing our life-world reasons for acting. 
These reasons are, of course, not only empirical but also 
normative. However, as participants in a specific form 
of life, we must take these reasons seriously. We can be 
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convinced by arguments that some of these life-world 
reasons are inadequate or incompatible with other rea- 
sons that we might consider to be more fundamental, but 
we can never leave the normative frame of the life form 
we participate in (Nida-Rimelin, 2009). The conceptual 
frame of TSR is better suited to integrate and systematize 
life-world practical reasons than conventional RCT. 
There are duties that relate to social roles, obligations 
that are related to personal commitments, and still others 
that originate from prescriptions or rights. The idea that 
all of these prima facie duties could be reduced to the nor- 
mative criterion of optimizing states of affairs, as in RCT, 
has a rationalistic turn of the worst kind. Compared to 
this, TSR is modest. It avoids philosophical and theoreti- 
cal hypocrisy; it takes practices and persons seriously but, 
nevertheless, preserves the mathematical core of decision 
theory (Nida-Rtimelin, 1997a). The agent acts rationally if 
the series of her singular acts constitutes an intended struc- 
ture (a pattern of actions) that she can accept as part of her 
form of life.* Philosophical or economic theory should not 
prescribe which practical reasons constitute a form of life. 


Notes 


The main body of the text was written by Julian Nida-Rtimelin, 
whereas Rebecca Gutwald and Niina Zuber surveyed the litera- 
ture and completed the text, focusing on alternative accounts of 
structural rationality in the contemporary debate. 


1. In fact, this term was introduced much earlier in German 
philosophy in the context of the critique of consequential- 
ism in ethics and rationality theory (cf. Nida-Rtimelin, 1993; 
accompanied by various articles in the 1990s such as Nida- 
Rumelin, 1991, 1994, 1997b, 1997c, 1997d). The core elements 
of structural rationality were later presented in an essay on 
structural rationality (Nida-Riimelin, 2001; English version: 
Nida-Rtimelin, 2019, Part I). It has been extended to a general 
theory of practical reason (Nida-Riimelin, 2020). 


2. Benjamin Kiesewetter undertakes a thorough analysis of 
structural irrationality and structural requirements of rational- 
ity in his The Normativity of Rationality (Kiesewetter, 2017). 


3. In fact, infinitely many of those functions can be trans- 
formed into one another by positive linear transformations. 


4. See also Korsgaard (1996) for an account of autonomy and 
rationality. 
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10.6 Adaptationism: A Meta-Normative Theory of Rationality 


Leda Cosmides and John Tooby 


Summary 


Human cognition is often compared—unfavorably—to 
normative theories of rationality from mathematics, 
logic, economics, or philosophy. But what justifies these 
normative theories as the proper standard for assessing 
the rationality of an evolved computational system? The 
adaptationist program in evolutionary biology provides a 
meta-normative theory of rationality that is appropriate 
for evolved organisms. The functional design of our cog- 
nitive architecture was built by natural selection, to solve 
information-processing problems that are strange, exact, 
and nonintuitive. Theories of adaptive function—task 
analyses of these problems—provide normative standards 
of good design for assessing the rationality of human cog- 
nition. With five case studies, we show how this approach 
can reveal sophisticated cognitive mechanisms that would 
otherwise remain undetected. At the same time, these 
cases illustrate pitfalls of studying reasoning and choice 
without reference to the ancestral problems and environ- 
ments that selected for their design. 


1. The Paradox of Human Reasoning 


How the mind works can be illuminated by comparing 
human cognition to standards of good design specified 
by a normative theory. But which standards are appro- 
priate for an evolved organism? What counts as a ratio- 
nal inference or choice for animals like us, whose minds 
were designed by natural selection? 

The first normative theories used in cognitive psy- 
chology were those deemed rational by mathematicians, 
scientists, economists, and philosophers. Deductive and 
inductive reasoning were compared to standards of ratio- 
nality drawn from first-order logic, Bayes’ theorem, or 
statistical principles. The manner in which people evalu- 
ate hypotheses was compared to Popper’s standard: look 
for violations, not confirmation. Decision making was 
compared to Neyman-Pearson decision theory, expected 


utility theory, or, when choosing among bundles of goods, 
the axiom of transitivity (e.g., GARP). 

In this research tradition, reasoning errors are defined 
as having occurred whenever human judgment departs 
from what these normative theories dictate. In their sem- 
inal studies of judgment under uncertainty, Kahneman 
and Tversky (1982) articulated this standard: “The pres- 
ence of an error in judgment is demonstrated by com- 
paring people’s responses either with an established fact 
(e.g., that the two lines are equal in length) or with an 
accepted rule of arithmetic, logic, or statistics” (p. 123). 

When cognitive performance was assessed by these 
normative standards, the rationality of human reason- 
ing, judgment, and decision making was found wanting. 
Psychologists found systematic errors in deductive rea- 
soning and hypothesis testing, violations of probability 
theory, preference reversals, and “excess” altruism. These 
“errors” were attributed to a flotilla of heuristics, biases, 
and emotions—cognitive processes thought to be fast, 
automatic, unconscious, emotional, associative, and/or 
stereotypic. This heterogeneous collection of processes was 
called “System 1.” Rational thinking was attributed to “Sys- 
tem 2”: slow, effortful, conscious deliberations that carry 
out calculations and logical inferences (Kahneman, 2011). 

This research program has produced a formidable par- 
adox. Reviews of human reasoning research are custom- 
arily presented as lengthy catalogs of errors, fallacies, 
biases, heuristics, and emotions. Textbooks organize 
their discussions around the seemingly endless ways in 
which human reasoning and decision making depart 
from the normative ideals of rationality used in science, 
mathematics, economics, and philosophy. Yet evolved 
reasoning systems—human and nonhuman minds 
alike—negotiate the complex natural tasks of their world 
with a level of operational success far surpassing that of 
the most sophisticated existing artificial intelligence (AI) 
systems. It is trivial to equip AI systems with algorithms 
that implement statistical decision theories, inferences 
of first-order logic, and other formal methods of rational 
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inference. They can search vast problem spaces in sec- 
onds, store massive databases for analysis, and perform 
lightning calculations. But organisms perform better than 
these computational systems on virtually every natural 
inferential problem that has been carefully investigated— 
the induction of grammar and word meanings, speech 
perception, vision, color constancy, recognizing objects 
and making inferences about their interactions, and infer- 
ring the beliefs, desires, and behavior of other people, to 
name a few. 

What is the resolution to this paradox? From the per- 
spective of evolutionary biology, the problem is not that 
our thinking is irrational. The problem is how psycholo- 
gists have been defining rationality and testing for its 
presence. 


2. A Meta-Normative Theory of Rationality 


Let us take one step back and ask a more fundamental 
question. Why identify rationality with adherence to 
normative theories from logic, mathematics, economics, 
and philosophy? What normative theory justifies the 
choice of these particular normative theories? 

In common parlance, a choice, process, or behavior is 
considered rational when it is well designed for achiev- 
ing a goal, and irrational when poorly designed for that 
function. It would be irrational to eat charcoal instead of 
fruit to sate hunger (but rational if you had just ingested a 
poison); it would be irrational to travel to a new location 
by randomly zigzagging instead of following a straight- 
line path (but rational if you are evading a predator); it 
would be irrational to converse out loud while alone (but 
rational if you are rehearsing a role). There is no goal- 
independent definition of what counts as rational. 

The specialized (and historically recent) tools of sci- 
entists, mathematicians, economists, and philosophers 
were developed for very specific goals: producing knowl- 
edge, understanding choice, and guiding behavior in 
mass societies. From the 1400s to the present, increas- 
ing literacy, more reliable data storage technologies, 
and the decreasing cost of communication and travel 
have changed the world beyond anything known by 
our hunter-gatherer ancestors. Because of these devel- 
opments, hypotheses and data became more widely 
shared, debated, vetted, and stored, enabling the cul- 
tural accumulation of knowledge in ever-larger popula- 
tions; markets arose in which millions of anonymous 
individuals cooperated in complex networks; people 
from locations with different moral norms were brought 
into contact, in ever-widening circles of interaction. 
Applying standards of rationality created by scientists, 
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mathematicians, economists, and philosophers is use- 
ful for solving the evolutionarily novel problems cre- 
ated by these conditions, such as producing scientific 
knowledge, understanding market dynamics, and, per- 
haps, fostering cooperation in the modern world of mass 
societies and global contact. But what justifies privileg- 
ing methods developed for these modern goals as the 
metric against which human rationality is measured? 

Nothing. From an evolutionary perspective, these are 
the wrong goals against which to measure human ratio- 
nality, because they played no causal role in selecting for 
the design of the mind. To decide whether a mechanism 
of human reasoning or decision making is rational— 
whether it is well engineered for producing a goal or 
outcome—we need to know what adaptive problem that 
mechanism was designed (by natural selection) to solve, 
and what kind of information was available for solving it 
in the environments that selected for its design. 


3. The Adaptationist Program in Evolutionary 
Biology Provides a Sound Meta-Normative 
Theory of Rationality 


From the molecular machinery of a single cell to the 
information-processing architecture of the human visual 
system, the most sophisticated engineering on Earth is 
found in organisms and was built by natural selection. 

Organisms are composed of many machines: sys- 
tems organized to solve a problem. These systems evolved 
to solve problems in ways that improved the ability of the 
organism’s ancestors to produce offspring. Every piece of 
organic machinery—the retina, the heart, the lungs— 
acquired its intricate functional design over deep time, 
as a downstream consequence of the fact that organisms 
reproduce themselves. 


3.1 Natural Selection Retains Designs That Promoted 
Their Own Reproduction 

When individuals reproduce, replicas of their organic 
machinery develop in their offspring. But replication 
is not error-free: chance mutations introduce changes 
into the design of organic machines, altering their fea- 
tures. These entropic changes usually disrupt the effi- 
ciency with which this machinery solves problems, 
thus interfering with the mutant offspring’s ability to 
produce offspring of its own. Because individuals with 
the newly modified, but now defective, design produce 
fewer offspring, on average, than those with the (more 
efficient) standard design, the defective design eventu- 
ally disappears from the population—a case of negative 
feedback. 


Adaptationism 


Occasionally, a mutation will improve a machine’s 
operation in a way that promotes the reproduction of indi- 
viduals with that mutation—and, therefore, the reproduc- 
tion of that design. Such improved designs (by definition) 
cause their own increasing frequency in the population—a 
case of positive feedback. This increase continues until 
(usually) the modified design out-reproduces, and thereby 
replaces, all alternative designs in the population, lead- 
ing to a new species-standard design. The population- or 
species-standard design has taken a step “uphill” toward a 
greater degree of functional organization for reproduction 
than it had previously. 

Over the long run, down chains of descent, this feed- 
back cycle—natural selection—pushes designs through 
state-space toward increasingly well-engineered func- 
tional arrangements. These arrangements are functional in 
a specific sense: the elements are well organized to cause 
their own reproduction in the environments in which the 
species evolved. 


3.2 Adaptive Problems and Evolved Solutions 
Enduring conditions in the world that create reproduc- 
tive opportunities or obstacles constitute adaptive prob- 
lems. Examples include the presence of predators, the 
possibility of sharing food to pool foraging risk, or the 
existence of a cognitive mechanism that could be repur- 
posed for more efficient foraging. Enduring relation- 
ships of this kind constitute reproductive opportunities 
or obstacles in the following sense: if the organism had 
a new property that interacted with these conditions 
in just the right way, then this property would cause 
individuals who have it to produce more offspring that 
live to reproductive maturity, relative to those with 
alternative designs. These reproductive opportunities 
and obstacles can be thought of as problems. A property 
is a solution to such a problem when it allows organ- 
isms with this property to take advantage of prevailing 
conditions, where “advantage” means a reproductive 
advantage. 

Adaptive problems have two defining characteristics. 
First, they are conditions or cause-and-effect relationships 
that many or most individual ancestors encountered, 
reappearing again and again during the evolutionary his- 
tory of the species. Second, they are that subset of endur- 
ing relationships that could, in principle, be exploited by 
some new property of an organism to increase its repro- 
duction or the reproduction of its relatives (who have a 
high probability of having inherited the same mutation). 

An enduring adaptive problem constantly selects for 
design features that promote the solution to the prob- 
lem. Over evolutionary time, more and more design 
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features accumulate that fit together to form an inte- 
grated structure or device that is well engineered to 
solve that adaptive problem. Such a structure or device 
is called an adaptation. An adaptation may have many 
beneficial effects, but solving the adaptive problem that 
selected for its design is its function. The function of 
the heart is to pump blood; the function of the liver 
is to detoxify poisons; the function of the retina is to 
detect photons and transduce them into neural signals 
for vision. Adaptive problems that required informa- 
tion processing for their solutions selected for neural 
circuitry organized to compute these solutions: compu- 
tational adaptations. 

Adaptationism is the name of the research program that 
explores how natural selection functionally organizes the 
designs of organisms. Adaptationists start with a careful 
analysis of an adaptive problem—including information 
that would have been available in ancestral environments 
for solving that problem. A careful task analysis of an adap- 
tive problem serves as a normative theory of good design: it 
identifies the problem, allowing one to see what counts as 
a good solution to it. Because a computational adaptation 
should be composed of algorithms and representations 
that are well engineered for solving the problem that 
selected for its design, a good theory of adaptive function 
suggests hypotheses about the design of the mind. These 
can be tested empirically, revealing computational opera- 
tions that were previously unknown. 

The adaptationist perspective provides principled, non- 
arbitrary criteria for judging rationality. If a choice, pro- 
cess, or behavior is rational when it is well designed for 
achieving a goal, then a computational adaptation that 
generates inferences or choices is rational when its features 
are well tailored for solving an adaptive problem faced by 
the hunter—gatherers from whom we are descended. 

The study of reasoning and choice looks very different 
when viewed from this perspective. Many fallacies dis- 
appear, and well-designed reasoning appears, when cog- 
nitive performance is compared to theories of adaptive 
function: normative standards of good design derived 
from evolutionary biology, behavioral ecology (of hunter- 
gatherers, nonhuman primates, and other animals), and 
studies of ancestral environments (from paleoanthropol- 
ogy, archaeology, physics, geology, botany, zoology, and 
other sources). Using five case studies of reasoning and 
choice, we illustrate pitfalls that arise from the failure 
to apply adaptationist thinking. Cases 3-5 tap theories 
of adaptive function that are normative in evolutionary 
biology but have no counterpart in traditional normative 
theories. 
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4. Case 1: To Discover a Well-Designed Reasoning 
Mechanism, You Need to Present Stimuli 
in an Ecologically Valid Format 


Behavioral ecologists have sophisticated models of the 
adaptive problems animals face when foraging. Because 
efficient foraging requires animals to make judgments 
under uncertainty, these models incorporate elements 
of probability theory. Behavioral ecologists typically find 
that insects, birds, and other animals behave like good 
Bayesians when they make foraging decisions (e.g., Real, 
1991; Stephens & Krebs, 1986). Yet psychologists thought 
that the human mind was “too limited” in capacity to do 
the same (Kahneman, Slovic, & Tversky, 1982). Why the 
difference? 

Computational adaptations should be ecologically 
rational: designed to work well in the ecological cir- 
cumstances that selected for their design. Consider, for 
example, how well your visual system maintains color 
constancy given normal variations in light cast by the 
sun: your green car looks green all day, even at sunset, 
when it is bathed in “red” (long-wavelength) light. Yet 
these sophisticated mechanisms fail—your green car 
looks brown—in the ecologically novel spectrum cast 
by sodium vapor streetlights. When psychologists first 
started to study Bayesian reasoning, the stimuli they 
used were the cognitive equivalent of sodium vapor 
lights: probabilities of single events (e.g., there is a 4% 
chance you will find an apple tree in this orchard) and 
normalized frequencies (36% of the trees are cherry 
trees). These data formats are the recent cultural product 
of modern data gathering and statistical techniques. 

When asked to compute a conditional probability 
based on such data (e.g., What is the chance that a tree 
with red fruit is an apple tree?), people fail spectacu- 
larly. But they succeed when given natural frequencies: 
the absolute frequencies of events as you encounter them 
in the world (Gigerenzer & Hoffrage, 1995, 1999). Our 
minds automatically encode natural frequencies, which 
were the only kind of probability data available in the 
ecology of our hominin ancestors (Cosmides & Tooby, 
1996). Imagine an orchard with 125 fruit trees—apple, 
cherry, lemon, and pear. On a stroll, you encounter 50 
trees with red fruit: 5 apple trees and 45 cherry trees. 
Given these natural frequencies, most people realize that 
5 out of 50 trees with red fruit are apple trees—the con- 
ditional probability that a tree with red fruit is an apple 
tree (hits / (hits + false alarms)). Note that the low base 
rate of apple trees (4%) is implicit in the small number 
of them that you encounter as you walk through the 
orchard (thereby taking a “natural sample”; Kleiter, 
1994). Adaptations for making probability judgments 
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can safely ignore normalized data about base rates (as 
people famously do) if they are designed to use natu- 
ral frequencies derived from natural samples. (On the 
fit between cognitive mechanisms and their task envi- 
ronment, see chapter 8.5 by Hertwig & Kozyreva, this 
handbook.) 

Our mental mechanisms do produce judgments that 
match normative standards from probability theory— 
when they are fed data in the format they were designed 
to use. The practical relevance for modern environments 
is clear: a simple change in how data are communicated 
can help patients and policy makers make rational deci- 
sions about risk (Gigerenzer, 2014). 


5. Case 2: Judgments That Look Like Normative 

Violations May Be Very Well Designed for Solving 
a Recurrent Adaptive Problem—One the Scientist 
Is Not Considering 


In choosing between risky and sure options, many 

people behave as if “losses loom larger than gains”: they 

are willing to take a bigger risk to avoid losing $100 than 
to gain the same amount. To appreciate the puzzle, imag- 

ine there is an outbreak of a disease that will kill 600 

people if nothing is done. Which of two programs to 

combat the disease would you favor? 

If program A is adopted, 400 people will die. If program 
B is adopted, there is 1/3 probability that nobody will 
die, and 2/3 probability that 600 people will die. (italics 
added) 

Expected utility theory predicts indifference, because 

both programs have the same expected value. With 

program A, 400 people die and 200 survive; program B 

produces the same result, on average. But Tversky and 

Kahneman (1981) found that people are not indifferent: 

when the options were framed as lives lost, most people 

(~ 80%) chose the risky option, program B. 

Even more puzzling: this preference reversed—most 
people (~ 70%) chose the sure option, program A—when 
the same options were framed as gains: 


If program A is adopted, 200 people will be saved. If pro- 
gram B is adopted, there is 1/3 probability that 600 
people will be saved, and 2/3 probability that no people 
will be saved. 


Why? The loss and gain frames express logically and 
mathematically identical situations. Preference reversals 
are considered irrational: they violate the transitivity 
axiom of expected utility theory. 

Because expected utility theory cannot explain these 
choices, psychologists and economists proposed that 
people have a stable taste: an aversion to loss. But a 
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normative theory from behavioral ecology explains 
these results—and correctly predicts when they will flip. 


5.1 Risk-Sensitivity Theory, an Alternative Analysis 

of Rational Decision Making 

Consider a bird who is deciding to forage on one of two 
patches. Both patches yield about 200 seeds per day of 
foraging—they have the same expected value—but they 
differ in variance. The low-variance patch yields close to 
200 seeds each day. The yield of the high-variance patch 
varies wildly from one day to the next, from 50 to 400 
seeds. 

Now consider the bird’s needs. If the bird needs to 
find 100 seeds today to live to tomorrow—a number 
below the expected value—the safest bet is to forage on 
the low-variance patch. But this is a bad bet if the bird 
needs 300 seeds to live another day: a rational bird will 
forage on the high-variance patch when its need is above 
the expected value. According to risk-sensitivity theory, 
a normatively correct decision takes three variables into 
account: (1) the expected value of each option, (2) the 
outcome variance associated with each option, and (3) 
the decision maker’s need level. A rational decision 
system will be designed to minimize the probability of 
an outcome that fails to satisfy one’s need (Stephens & 
Krebs, 1986). 

To test risk-sensitivity theory in humans, the experi- 
menter must vary the minimum need level before sub- 
jects make a decision. When no need level is specified, 
the subject is free to fill in the blank any way she wishes. 
For monetary gambles, maintaining the status quo is as 
good a minimum need level as any; a rational individual 
who does not want to fall below the status quo will avoid 
the risky option and choose the sure one. But this is not 
because she has an aversion to loss—a stable taste—that is 
independent of context. 

Like birds, people choose rationally when their min- 
imum need level is specified (Mishra & Fiddick, 2012; 
Rode, Cosmides, Hell, & Tooby, 1999). When it is above 
the expected value, people prefer the risky option. They 
choose the sure/low-risk option—thereby appearing 
to be loss averse—only when they need less than the 
expected value. Framing affects choice on the disease 
problem by changing the minimum number of lives 
people feel they must save: saying that 400 people will 
die leads people to set a higher threshold than saying 
that 200 will survive (Mishra & Fiddick, 2012). As a 
bonus, the same theory shows that “ambiguity avoid- 
ance” is also a myth (Camerer & Weber, 1992). Ambi- 
guity is usually interpreted as “risky,” but when the 
context implies it is the lower-variance option, people 
prefer the ambiguous option (Rode et al., 1999). 
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A tangle of results that look irrational when com- 
pared to traditional normative theories turn out to be 
rational responses to a problem that scientists were not 
considering. 


6. Case 3: Cues of a Reasoning System's Proper 
Domain—the Context for Which It Evolved—May 

Be Necessary to Activate Its Procedures. To Know Its 
Proper Domain, One Needs to Correctly Characterize 
the System’s Adaptive Function 


An adaptation’s proper domain is the information that the 
mechanism was designed by natural selection to process 
(Sperber, 1994). Ground squirrels, for example, evolved 
to produce alarm calls when they see a hawk overhead. 
The sight of a hawk is a cue that the squirrel is in danger 
from raptors now. This cue elicits the correct alarm call 
from the squirrel’s repertoire (snakes and predatory cats 
elicit different calls). The approach of predatory birds is 
the adaptation’s proper domain. 

The raptor call is also activated when zoologists fly a 
drone with the silhouette of a hawk overhead. The alarm 
call’s actual domain is larger than its proper domain: 
it consists of all cues that activate the call, real hawks 
(proper domain) and hawk silhouettes alike. In studying 
a reasoning mechanism in humans, experimenters need 
to consider (i) the mechanism’s evolved function, to 
ascertain its proper domain, and (ii) whether the experi- 
mental stimuli present cues from its actual domain— 
ones sufficient to activate the mechanism. Mercier and 
Sperber (2011) have argued that most studies finding 
poor logical reasoning lack cues to the mechanism’s 
proper domain: devising and evaluating arguments 
intended to persuade or dissuade another person. 

Eliciting poor logical reasoning from people is easy: 
give them a problem, without context, that requires them 
to produce a valid inference using modus tollens (MT). 
MT is an inference rule from first-order logic. Given a 
conditional rule, such as “If the pipe was fixed, then the 
bathroom floor will be dry in the morning,” and a prem- 
ise, such as “The bathroom floor is wet this morning,” it 
is logically correct to infer “The pipe was not fixed” (more 
generally: If P then Q; not-Q; therefore not-P). It is trivial 
to program a computer to produce an MT inference. Yet 
people fail to do so about 75% of the time (Rips, 1994; 
Wason & Johnson-Laird, 1972). 

But do our minds lack an MT inference rule? Or do 
indicative conditional rules stripped of context fail to 
activate it? The mind does seem to implement certain 
rules of first-order logic (e.g., people usually apply the 
modus ponens rule, correctly inferring Q from If P then 
Q and P). If the human mind has a system designed for 
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reasoning logically, what adaptive problem did it evolve 
to solve? Mercier and Sperber (2011) have proposed 
that conscious, deliberative reasoning—including logi- 
cal reasoning—evolved to solve adaptive problems that 
arise during communication. 


6.1 Biologists Have a Normative Theory 

of Communication 

A seminal paper on the evolution of communication by 
Krebs and Dawkins (1984) changed how biologists study 
animal communication. A communication system will 
not evolve unless it confers a net benefit on both senders 
and receivers—if a signal benefits only senders, receivers 
will not evolve mechanisms to decode it. But once a sig- 
naling system has evolved, situations will arise in which 
senders can manipulate receivers, to their own advan- 
tage, by sending deceptive signals. If acting on deceptive 
signals is costly to the fitness of the receiver, selection 
will favor adaptations in receivers to detect which sig- 
nals are deceptive. As receivers get better at distinguish- 
ing honest from deceptive signals, selection will favor 
adaptations in senders for producing deceptive signals 
that are more difficult to detect, and so on. A coevo- 
lutionary arms race ensues over generations: receivers 
get better and better at detecting deceptive signals, and 
senders get better and better at making deceptive signals 
resemble honest ones. By contrast, the fitness interests 
of receivers and honest signalers converge; this favors 
the evolution of honest signals that are difficult for a 
deceptive sender to fake (Higham, 2014). 

Language presents additional problems, however, 
because sending a deceptive message is no more costly 
(in words produced) than sending an honest one. 
Detecting the veracity of a signal requires logical reason- 
ing, according to Mercier and Sperber, so that receivers 
can maintain “epistemic vigilance.” While the receiver 
is evaluating the sender’s claim, it is stored in a special 
data format—a meta-representation—which is decou- 
pled from the receiver’s semantic memory (her database 
of knowledge). Logical reasoning is necessary for her to 
detect whether the claim contradicts other claims made 
by the sender, claims made by other people, and facts 
she already knows. This evaluation process is particu- 
larly important, they claim, in arguments intended to 
persuade the receiver to act on beliefs she does not yet 
hold or dissuade her from actions she wants to take. 


6.2 Argumentative Context: A Cue That Activates 
Logical Reasoning 

When the sender and receiver already agree, there is 
no argument—no attempt to persuade or dissuade. If 
they disagree and an argument ensues, there are two 
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possibilities: (i) one (or both) of them are misinformed 
(and they need more accurate information), or (ii) a 
deceptive message is being sent, intended to manipulate 
the receiver into doing something against her interests 
that benefits the sender. Both call for epistemic vigilance 
by the receiver: the claims must be evaluated for their 
truth value. The presence of an argument is, therefore, a 
cue that should activate logical reasoning in the receiver. 
It should also activate logical reasoning in the sender, 
who must devise an argument that will persuade the 
receiver. 

This has empirical implications. Divorced from the 
context of an argument, the MT inference may not be 
activated. But it can be easily activated in an argumenta- 
tive context. Let’s say Iam arguing with the plumber, who 
claims that he fixed the leaky pipe in my powder room. 
I doubt him, and deny his claim by saying, “Oh yeah? 
We'll see! If the pipe was fixed, then the bathroom floor will 
be dry in the morning.” When I discover that the bathroom 
floor is wet in the morning, we will both conclude that 
the pipe was not fixed: we easily make the MT inference. 

Mercier and Sperber review evidence in favor of their 
hypothesis that an argumentative context is the proper 
domain for reasoning, logical and otherwise. Their claim 
is not that reasoning will be flawlessly logical in all argu- 
ments: an argumentative context will activate logic 
when the reasoner must be epistemically vigilant, but 
confirmation bias and other infelicities will emerge when 
sound reasoning would undermine one’s attempt to per- 
suade. In their words, “In all these instances tradition- 
ally described as failures or flaws, reasoning does exactly 
what can be expected of an argumentative device: Look 
for arguments that support a given conclusion, and, 
ceteris paribus, favor conclusions for which arguments 
can be found” (Mercier & Sperber, 2011, p. 57). 


7. Case 4: A Cognitive Adaptation for Reasoning May 
Be Specialized for a Specific Domain and, Therefore, 
Equipped with Procedures That Are Content Rich 
Rather Than Content Free 


Normative theories of rationality from logic, mathematics, 
and economics typically posit content-free reasoning pro- 
cedures: ones that operate uniformly on information from 
every domain. Such procedures—domain-general ones—do 
exist in the human mind (automatic frequency computa- 
tion is an example), but they cannot solve even routine 
adaptive problems by themselves (Cosmides & Tooby, 
1987; Tooby & Cosmides, 1992). Domain-specialized rea- 
soning systems were also required. These are equipped 
with content-rich concepts, inference procedures, and 
decision rules, ones that are superbly engineered for 
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producing fitness-promoting inferences in one ancestral 
domain but do not apply outside it (e.g., concepts like 
belief and desire are useful for predicting the behavior of 
people, but not rocks). Discovering these systems requires 
careful analysis of an adaptive problem to see what counts 
as a functional (i.e., normative) solution. 

The evolution of cooperation for mutual benefit— 
social exchange—poses exacting problems, which have 
been modeled extensively. In humans, these problems 
are solved by a functionally specialized reasoning sys- 
tem that deploys content-rich representations and pro- 
cedures. Its features were revealed by experiments that 
tested hypotheses derived from evolutionary game 
theory—hypotheses that were constructed in advance of 
collecting any data. 


7.1 Evolutionary Game Theory Is a Source 

of Normative Theories 

Game theory is a tool for analyzing strategic social 
behavior—how agents will behave when they are inter- 
acting with others who can anticipate and respond to 
their behavior (chapter 9.3 by Alexander, this hand- 
book). Economists use it to analyze how people respond 
to incentives present in the immediate situation. Their 
models typically assume rational actors, who calculate 
the payoffs of alternative options (anticipating that 
other players will do likewise) and choose the option 
most likely to maximize their short-term profits (but see 
Hoffman, McCabe, & Smith, 1998). 

Evolutionary biologists also adopted game theory as 
an analytic tool, but with a twist (Maynard Smith, 1982). 
Evolutionary game theory does not assume economically 
rational agents who can reason about the reasoning of 
other agents via “backward induction.” It can be use- 
fully applied to cooperation among bacteria or fighting 
in spiders. It is used to model interactions among agents 
endowed with well-defined decision rules that produce 
situationally contingent behavior. Although these deci- 
sion rules are sometimes called “strategies” by evolution- 
ary biologists, no conscious deliberation by bacteria (or 
humans) is implied (or ruled out) by this term. Sometimes 
results are derived analytically; in more complex cases, 
agent-based simulations of natural selection are used. 

Whether the decision rules being analyzed are 
designed to regulate foraging, fighting, or cooperating, 
the immediate payoffs of these decisions, in food or 
resources, are translated into the currency of offspring 
produced by the decision-making agent, and these off- 
spring inherit their parent’s decision rule. In evolution- 
ary game theory, a decision rule or strategy that garners 
higher payoffs leaves more copies of itself in the next 
generation than alternatives that garner lower payoffs. By 
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analyzing the reproductive consequences of alternative 
decision rules over generations, evolutionary biologists 
can determine which strategies natural selection is likely 
to favor and which are likely to be selected out. This source 
of normative theories can be tapped for any domain in 
which organisms make consequential choices. 

The evolution of cooperation has been extensively 
modeled using these tools, with clear results. Strategies that 
indiscriminately provide benefits to others—including 
those who do not reciprocate—are eventually eliminated 
from the population. Because they incur reproductive 
costs without compensating benefits, they are outcom- 
peted by designs that cheat—that accept benefits with- 
out reciprocating them. Selection favors strategies that 
cooperate conditionally: ones that cooperate with other 
cooperators and withdraw cooperation from cheaters 
(e.g., Axelrod, 1984; Trivers, 1971). A cheater is an agent 
endowed with decision rules that accept benefits offered 
by other agents without satisfying their requirements. 
Innocent mistakes do not reveal a cheater; a cheater is 
an agent who violates a social exchange agreement by 
design (by virtue of its decision rules), not by accident. 

This analysis carries many implications about how a 
system must be designed to implement a strategy for con- 
ditional cooperation. The most straightforward: for social 
exchange to evolve, agents must have mechanisms that 
make them very good at searching for information that 
would reveal cheaters. A content-free logic, deontic or 
otherwise, cannot do the job (e.g., Cosmides & Tooby, 
2008). 


7.2 Information Search: Looking for Cheaters 
versus Looking for Logical Violations 
The philosopher Karl Popper proposed a normative stan- 
dard for evaluating hypotheses: look for violations, not 
confirmation. “All swans are white” is violated by finding 
a single black swan, no matter how many white swans 
you have so far observed. (Realizing this requires the MT 
inference.) Peter Wason developed his four-card selection 
task to find out if people are natural falsificationists, who 
look for cases that could violate a hypothesis that is pre- 
sented as a conditional rule (/f P then Q). His research 
suggests that we are not (Wason & Johnson-Laird, 1972). 
Imagine, for example, that you are given incomplete 
information about four birds—two of unknown color (a 
swan and a parrot) and two of unknown species (one 
black and one white). Which birds should you investi- 
gate further to see if any of them violate the rule “If a 
bird is a swan, then it is white”? Most people want to 
investigate the swan, to learn its color. That is logically 
correct: discovering a black swan would violate the rule. 
But that implies you should also investigate the black 
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bird, to learn its species: it too might be a black swan. Most 
people fail to investigate the black bird, and many want 
to investigate the white one (unnecessary: white birds— 
swan, dove, parrot—cannot violate the rule). Only cases 
of P & not-Q (here, swans that are not white) can violate If 
P then Q. Most people recognize this when asked, but they 
do not spontaneously use logic to search for violations of 
indicative rules. Fewer than 25% of people seek informa- 
tion about P, not-Q, and no other case. 

By contrast, 65% to 80% of people successfully look 
for violations—cheaters—when the rule involves social 
exchange. The mind interprets a conditional rule as a 
social contract when it expresses an agreement to cooper- 
ate for mutual benefit, for example, “If you borrow my 
car, then you must fill the tank with gas” (or, more gen- 
erally, “If you accept benefit B from agent J, then you 
must satisfy J’s requirement R”). It becomes obvious that 
one needs to investigate the guy who borrowed the car 
(P) and the one who did not fill the tank (not-Q). 


7.3 A Content-Rich Adaptive Logic 

Wason tasks involving social exchange activate a cog- 
nitive adaptation that evolved for detecting cheaters. It 
is part of a computational system that is specialized for 
reasoning about social exchange (for a review of the evi- 
dence, see Cosmides & Tooby, 2015). The social exchange 
system dissociates, both functionally and neurally, from 
reasoning systems that are activated by content tapping 
other domains (including precautionary rules, which 
are so similar to social contracts that most theories do 
not distinguish them). It represents social contracts 
using content-rich proprietary concepts (e.g., agent;, ben- 
efit to agent;, requirement of agent;, obligation, entitlement, 
cheater). Its procedures operate on these representations, 
producing inferences appropriate to social exchange 
that are not licensed by content-free logics (deontic or 
otherwise). Its cheater detection mechanism attends to 
information that would reveal cheaters, whether the 
resulting answer is logically correct or not. And that 
mechanism looks for cheaters—innocent mistakes do 
not elicit violation detection. The design features of this 
content-rich system are normatively correct: they are 
precisely tailored for their adaptive function. 


8. Case 5: Choices That Deviate from Economic 
Rationality May Be the Most Adaptive, 
Fitness-Promoting Strategy—Not a “Bias” 


Economists have built models in which rational indi- 
viduals and firms behave “as if” they were maximizing 
profits (or, more accurately, utility functions). A utility 


Leda Cosmides and John Tooby 


function is a mathematical device for understanding the 
dynamics of markets in which millions of anonymous 
individuals cooperate for mutual benefit: it transforms 
changes in inputs (e.g., the price of corn) into changes 
in outputs (e.g., the quantity of tortilla chips produced). 
These models are successful for their intended purpose. 
But they fail when they are used to predict the behav- 
ior of individuals cooperating in small groups (Smith, 
2003). Is this another failure of human rationality? 


8.1 A Puzzle from Behavioral Economics 

Cooperation can be studied in the laboratory by having 
people interact in games in which the monetary payoffs 
for different choices are carefully controlled—dictator 
games, prisoner’s dilemma games, bargaining games 
(e.g., the ultimatum game), trust/investment games, 
public goods games, and others. When behavioral econ- 
omists used these methods to test predictions of game 
theory, they found that people in small groups do not 
act as if they are maximizing immediate monetary pay- 
offs (e.g., Hoffman et al., 1998). In a one-shot interac- 
tion with anonymous others, Homo economicus models 
predict no generosity, no cooperation, no trust, and 
no punishment. Yet people give more, cooperate more, 
trust more, and punish defections more than these mod- 
els predict, even when the experimenter tells them that 
the interaction is one-shot and anonymous. But why? 
Generosity in anonymous, one-shot games is irrational, 
given the standard theories. According to both economic 
and evolutionary game theory, repeated interactions are 
necessary for behaviors like this to evolve. 

This “excess altruism” is considered irrational on 
many economic theories, and some social scientists have 
viewed it as evidence that the psychology of coopera- 
tion was shaped by group selection rather than selection 
operating on individuals (e.g., Bowles & Gintis, 2013). 
But are these behaviors really excess altruism—that is, 
beyond what can be explained by selection on individu- 
als for direct reciprocity? 


8.2 Adaptationist Game Theory: Model the 
Information-Processing Problem and Let the 
Psychology Evolve in Response 

Selection does not occur in a vacuum: the physical and 
social ecology of a species shapes the design of its adapta- 
tions, and our hunter-gatherer ancestors lived in small, 
interdependent bands, and had many encounters with 
individuals from neighboring bands. Adaptations for 
direct reciprocity evolved to regulate cooperation in an 
ancestral world in which most interactions were repeated. 
The high prior probability that any given interaction will 
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be repeated should be reflected in their design. So should 
the fact that you have interacted at least once with a per- 
son: models of this social ecology show that meeting an 
individual once is itself a good cue that you will meet 
again (Krasnow, Delton, Tooby, & Cosmides, 2013). 

Whether an interaction is one-shot or repeated is 
a judgment made under uncertainty. What decision 
rules does selection favor when this judgment-under- 
uncertainty problem is modeled? 


8.3 The Evolution of Generosity under Uncertainty 

In most agent-based simulations of the evolution of 
cooperation, the behavioral strategies are particulate— 
they do not have internal cognitive components that 
can evolve—and the simulation environment has either 
one-shot or repeated interactions, but not both. But 
what happens if these strategies have a cognitive archi- 
tecture that can evolve, and the social environment 
includes both one-shot and repeated interactions, as in 
real life? It turns out that generosity in one-shot inter- 
actions evolves easily when natural selection shapes 
decision systems for regulating two-person reciprocity 
(exchange) under conditions of uncertainty (Delton, 
Krasnow, Cosmides, & Tooby, 2011). 

In real life, you never know with certainty that you 
will interact with a person once and only once. Catego- 
rizing an interaction as one-shot or repeated is always 
a judgment made under uncertainty, based on probabi- 
listic cues (e.g., Am I far from home? Did he marry into 
my band?). In deciding whether to initiate a cooperative 
relationship, a contingent cooperator must use these 
cues to make trade-offs between two different kinds of 
errors: (i) false positives, in which a one-shot interaction 
is mistakenly categorized as a repeated interaction, and 
(ii) misses, in which a repeated interaction is mistakenly 
categorized as one-shot. A miss is a missed opportunity 
to harvest gains in trade from a long string of mutually 
beneficial interactions. In a population of contingent 
cooperators, the cost of a miss is usually much higher 
than the cost of a false positive. 

Using agent-based simulations, Delton et al. (2011) 
showed that, under a wide range of conditions, individual- 
level selection favors computational designs that decide 
to cooperate with new partners, even in a world where 
most of the interactions are one-shot. Each new partner 
comes with a number—a cue summary—that serves as 
a hint to whether an agent’s interaction with that part- 
ner will be one-shot or repeated. The cue summaries are 
never perfect predictors: they are drawn from one of two 
normal distributions (one-shot vs. repeated) that overlap 
by different amounts. 
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In one set of simulations, agents evolve a decision 
threshold determining how strong cues that the interac- 
tion is one-shot must be before the agent defects. Selection 
favored a threshold of evidence so high that most interac- 
tions were classified as repeated, triggering cooperation. In 
other simulations, the agents were perfect Bayesians who 
developed rational beliefs about whether an interaction is 
one-shot by using Bayes’ rule to integrate (i) cues that the 
given interaction is one-shot, with (ii) perfect knowledge 
of the base rate of one-shot interactions in the population. 
What evolved is a regulatory variable that determines the 
probability the agent will cooperate given its rational belief 
that the interaction is one-shot. 

Selection favored designs with a very high probability 
(70%-90%) of cooperating given the rational belief that 
the interaction is one-shot, with modest gains in trade and 
a modest number of encounters for those interactions that 
were repeated. This was true even when the base rate of 
one-shot interactions was unrealistically high (S0%-70%). 

The simulations with Bayesian agents are particularly 
apt because most subjects who cooperate in experimen- 
tal economics games say they believed the experimenter’s 
claim (a cue!) that their interaction would be one-shot. 
The results show that natural selection can favor a disposi- 
tion to start out cooperating, even in people who ratio- 
nally believe an interaction is most likely to be one-shot. 
No group selection is needed. And this disposition to 
cooperate in one-shot interactions is not a mistake or an 
irrational “bias”: it is the most adaptive, fitness-promoting 
decision. 
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Section 11 Deontic and Legal Reasoning 


11.1. Deontic Logic 


John Horty and Olivier Roy 


Summary 


Deontic logic is a general area encompassing many spe- 
cific logical approaches united by a common concern 
with normative concepts—concepts, that is, involving 
norms, such as moral, legal, or rational norms. Based 
on normative information of this kind, deontic logics, 
most typically, provide judgments or conclusions about 
what we ought to do and what we are permitted to do, 
or about what ought to be the case. In this chapter, we 
sketch the basic ideas underlying the standard system of 
deontic logic. Next, just to highlight the interdisciplin- 
ary reach of deontic logic, we mention some connections 
with game theory that we believe may be of particular 
interest to readers of this volume. Finally, we survey 
some important variations on standard deontic logic. 


1. What Is Deontic Logic? 


Deontic logic is a general area encompassing many spe- 
cific logical approaches united by a common concern 
with normative concepts—concepts, that is, involving 
norms, such as moral, legal, or rational norms; the norms 
of etiquette or aesthetics; or the rules of a game. Based 
on normative information of this kind, what deontic log- 
ics provide, most typically, is judgments or conclusions 
about what we ought to do and what we are permitted to 
do, or about what ought to be the case. 

Like many areas of logic, deontic logic was studied in 
a fragmentary way by medieval theorists, but it was not 
until the 20th century, with the introduction of mathe- 
matical techniques into logic, that the subject developed 
into a systematic discipline. This development can be 
divided into three stages. In the first, beginning with the 
work of Mally (1926) early in the 20th century, a number 
of philosophers and logicians explored ways of extend- 
ing the axiomatic approach to logic, in particular that 
of Whitehead and Russell’s Principia Mathematica (1910- 
1913), to deontic concepts as well. Although these early 
axiomatizations were soon shown to be flawed, they 


opened up the area for investigation and inspired later 
researchers. The second stage began near the midpoint 
of the 20th century with von Wright’s (1951) interpreta- 
tion of deontic logic as a species of modal logic, allowing 
for the subject to be studied using semantic techniques, 
and leading, with only slight modifications, to the sys- 
tem of modal logic now known as “standard deontic 
logic.” Although this system stimulated some interest- 
ing technical developments, it also served as the primary 
focus of a philosophical debate concerning the relation 
between deontic principles and moral concepts that 
became, over time, increasingly arid and scholastic. 

The third stage in the development of deontic logic, 
which began in the final decade of the 20th century and 
continues to this day, has been characterized by increas- 
ing interdisciplinary connection with many other fields, 
including artificial intelligence, multiagent systems and 
computer science more generally, legal theory, linguistics, 
political science, organizational theory, and economics. 
These interdisciplinary connections have revitalized the 
field, leading not only to new areas of application for 
ideas from deontic logic but also to the incorporation of 
new techniques into the study of deontic logic itself. 

We cannot attempt here either to trace the historical 
development of deontic logic in any detail or to survey 
its current range of interdisciplinary connections; an 
authoritative historical treatment is presented by Hilpinen 
and McNamara (2014). Our discussion is organized as fol- 
lows: first, we sketch the basic ideas underlying the stan- 
dard system of deontic logic. Next, just to highlight the 
interdisciplinary reach of deontic logic, we mention some 
connections with game theory that we believe may be of 
particular interest to readers of this volume. Finally, we 
survey some important variations on standard deontic 
logic. 


2. Standard Deontic Logic 


Standard deontic logic is a species of modal logic, a form 
of logic in which the truth or falsity of a formula in a 
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particular state of affairs can depend on the truth value 
of different formulas in different states of affairs. These 
states of affairs are often referred to, somewhat poeti- 
cally, as “possible worlds,” although the logic itself does 
not involve any metaphysical commitments, and the 
possible worlds can be provided with more concrete 
interpretations, as we shall see. 

Let us suppose, then, that W is a set of possible worlds 
and that v is a valuation function mapping each atomic 
sentence into the set of possible worlds where it is true, 
so that, where p is an atomic sentence and w is a possi- 
ble world, w is in v(p) if p is true in w, and w is not in 
v(p) if p is false in w. In addition to W and v, the struc- 
tures against which formulas of standard deontic logic 
are interpreted contain a further, distinctive component: 
a function f mapping each possible world w into a non- 
empty set f(w) of worlds that can be thought of as ideal 
from the standpoint of w. The basic idea underlying 
standard deontic logic is that what ought to be the case 
in the world w can be identified with what is the case 
in those worlds that are ideal from the standpoint of w. 

To develop the theory formally, we first introduce 
models of the form M = QW,f,v[], with W, f, and v defined 
as above. Next, letting M,wr A indicate that the for- 
mula A is true at the world w from the set W of worlds 
belonging to the model M, and where v is the valuation 
function from this model, we define truth conditions for 
atomic sentences through the clause 


M,w F p just in case w Ev(p), 


according to which the atomic sentence p is true at the 
world w just in case the valuation function says it is. Once 
the notion of truth at a world is settled for atomic sen- 
tences, it can be lifted to complex sentences formed from 
the connectives =, a, v, and ~—representing negation, 
conjunction, disjunction, and implication, respectively— 
through the clauses 


M,w £ WA just in case it is not the case that M,w E A, 
M,wetAa~B just in caseM,wkE A and M,we B, 
M,weEA v B just in case M,wkE A or M,wFE B, 
M,w — A-B just in case either not M,w E A, or 

M,we B. 


What these further clauses tell us is simply that the truth 
value assigned to a complex sentence at a possible world 
respects the meaning of the connective through which 
that complex sentence is formed: according to the sec- 
ond clause, for example, a conjunction is true at a world 
just in case each of its conjuncts is; according to the 
fourth, an implication is true just in case the consequent 
of that implication is true whenever its antecedent is. 
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Finally, we turn to the characteristic deontic connec- 
tive O, allowing for the construction of sentences of the 
form OA, to be read “It ought to be the case that A.” The 
clause governing this connective is 


M,wF OA just in case 
M,w' A for all w’ belonging to f(w), 


where of course fis the function from M mapping each 
world w into the set of worlds that can be considered as 
ideal from the standpoint of w. 

The clause governing the truth value assigned to a 
deontic statement captures in a formal way the idea that 
OA holds at a world w just in case A holds at all the worlds 
w’ that are ideal from the standpoint of w. For a concrete 
illustration, let A represent the statement that there is no 
war, and suppose we can agree that, from the standpoint 
of our actual world, any ideal world would have to be one 
in which there is no war, so that A holds in each of these 
ideal worlds. Then, what our logic tells us is that, because 
A holds in each of these ideal worlds, OA is true in the 
actual world—it ought to be that there is no war. 

One natural question to ask at this point is why the 
worlds considered to be ideal should be relativized to an 
initial world: why isn’t there just a single set of ideal 
worlds? The answer is that, in general, ought-statements 
can be contingent, varying from world to world. Con- 
sider, for example, the proposition that Jo promises to 
meet her friend Jack for dinner. Presumably this is a con- 
tingent proposition, holding at some worlds but not at 
others. So imagine that w is a world at which Jo makes 
this promise and w’ is a world at which she does not, and 
let A stand for the proposition that Jo, in fact, meets Jack 
for dinner—that she keeps her promise. If we assume 
that, ideally, promises are to be kept, then it follows that 
all the worlds ideal from the standpoint of w are worlds 
in which A holds, so that OA holds at w. However, since 
there is no particular reason to suppose that A holds at 
all the worlds ideal from the standpoint of w’, where Jo 
made no promise, it follows that OA fails at that possi- 
ble world. 

Once we have defined truth at a possible world in a 
model, we can, following the standard pattern in modal 
logic, define logical implication by stipulating that A 
logically implies B if B is true at every world from every 
model at which A is true. To illustrate, we can see that 
OA and O(A —B) implies OB, as follows. Suppose OA 
and O(A — B) are true at w. Then both A and A ~ B must 
be true at each world from f(w). But it follows from ordi- 
nary logic that B is true whenever A and A — B are both 
true, so that B must likewise be true at each world from 
f(w). From this, we can conclude that OB is true at w. 
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3. An Interlude: Connections with Game Theory 


Although the talk of possible worlds in deontic logic, 
and in modal logic more generally, can seem hopelessly 
abstract, it is important to realize that this language can 
be provided with concrete interpretations in a number 
of application domains. We cannot describe these vari- 
ous application domains here, of course, but just for the 
sake of illustration, we briefly sketch how the machinery 
of deontic logic can be applied in the domain of game 
theory, where the norms at work are not moral or legal 
norms but norms of rationality. Our discussion proceeds 
by way of example; see chapter 9.1 by Albert and Kliemt 
(this handbook) for formal definitions. 

Let us thus start with a simple example of a game in 
so-called normal or strategic form: 


Ann \Bob L R 
U alee 0,0 
D 0, 1 1,0 


In this game there are two players, Ann and Bob. Ann 
can choose the upper (U) or the lower (D) row and Bob 
either the left (L) or the (R) column. Each combination 
of these choices, for instance, (U, L) or (D, R), corresponds 
to one possible outcome of the game. Ann and Bob have 
preferences over these outcomes, which are expressed by 
the utility numbers in each cell of the matrix, with 1 
preferred to 0. The left-hand numbers are Ann’s utilities 
and the right-hand ones Bob’s. 

What is rational for Ann and for Bob to do in this 
game? What Ann should rationally do depends on what 
Bob does. If he plays L, then her best response is to 
play U. If, however, he plays R, then her best response 
is D. What is the best response for Bob, however, does 
not depend on what Ann does. If she plays U, he gets 1 
by playing L and 0 by playing R, and the same holds if 
Ann plays D. Game theorists would say that R is strictly 
dominated by L. Ruling R out as a rational choice for 
Bob, we conclude that he ought to play L. But then, as 
we have seen already, if we only consider L as a possible 
action for Bob, the only rational choice for Ann is to 
play U, leaving (U,L) as the only rational solution of this 
game. In other words, Ann ought to play U, and Bob L. 
This combination of choices also happens to be a Nash 
equilibrium, meaning that neither Ann nor Bob have 
an incentive to unilaterally deviate from it. Bob has no 
incentive whatsoever to play R instead of L. Ann, given 
that Bob plays L, should not switch to D either. 

Let us now look at how deontic logicians would ana- 
lyze this situation. First, we need to define the set W of 
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possible worlds. In our example, we can take W to con- 
sist of the four possible outcomes of the game, that is, 
W = {(U,L), (U,R), (D,L), (D,R)}. Second, we need to define 
the valuation function v. The atomic sentences that we 
are interested in describe the strategies that Ann and Bob 
can choose. Let us use u, d, |, and r for that. The valuation 
is then defined as the natural one: v(u) = {(U,L), (U,R)}, 
v(I) = {(U,L), (D,L)}, and so on. The final component that 
we need to define is the function fmapping each possible 
world w onto its set of ideal worlds. Here, again, there is 
simple, although of course not a unique, way to define 
this function f: we can assign the set of rational solutions, 
{(U,L)}, as the set of ideal worlds for all the four possible 
worlds in W. That is, for all w EW, f(w) = {(U,L)}. 

With this in hand, we can see what kind of deontic 
statements are true in the model that we have just con- 
structed. Recall that u is the statement that Ann plays U. 
Now, (U,L) is the only ideal world from the standpoint 
of any world in the model that we constructed above. 
In that world, u is true, so Ou is true everywhere in the 
model that we have constructed. Similarly, we have 
Ol true everywhere, reflecting the fact that in all ideal 
worlds, Bob plays L. 

Now, as we already observed, obligations and permis- 
sions might be contingent, varying from world to world. 
This is not the case in the model we just built. By setting 
f(w) = {(U,L)} everywhere, we have identified the set of 
ideal worlds with the unique outcome that results from 
first ruling out R as a rational choice for Bob and then iden- 
tifying Ann’s best response given that Bob does not play R. As 
we have seen, however, Ann’s best response would be dif- 
ferent if Bob would play R after all. To reflect that, we could 
instead define fas follows: f(U, L) = {(U,L)}, f((U,R) = {(D, R)}, 
f(D,L) = {(U,L)}, f(D, R) = {(D, L)}. With this modification, 
Ou becomes a contingent formula, true only in the possi- 
ble worlds where Bob plays his only rational strategy L but 
false in the nonideal case where Bob plays R. 


4. Variations 


The standard deontic logic sketched earlier is a starting 
point for the study of deontic logic, but only a starting 
point—this standard theory has been elaborated along 
various dimensions, sometimes in response to difficul- 
ties or perceived difficulties, and a variety of alternative 
frameworks have been explored. We cannot hope even 
to mention all of these developments in this brief chap- 
ter but will merely list a few of the most important. 
First, there are clear connections between deontic log- 
ics and logics of time, or temporal logics. Suppose, as in 
our earlier example, that Jo has promised to meet Jack 
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for dinner and so ought to have dinner with Jack. But 
of course, this event must occur at some particular point 
in the future, relative to the moment of the promise—it 
will do no good if Jo had dinner with Jack sometime in 
the past. Further, if Jo’s promise is to function as a real 
constraint on her behavior, we must at least be able to 
envision alternative futures, or alternative ways in which 
the world might unfold, in which Jo does not meet Jack 
for dinner. These issues concerning the temporal aspects 
of ought-statements have been explored in a number of 
papers beginning with seminal work in the early 1980s 
by Aqvist and Hoepelman (1981), Thomason (1981), and 
van Eck (1982); a useful survey can be found in Thoma- 
son (1984). Interestingly, similar issues arise in the study 
of games as well, as illustrated, again, by chapter 9.1 by 
Albert and Kliemt (this handbook). 

Second, deontic logics interact with issues of agency. 
Consider again our example, leading to the conclusion 
that Jo ought to dine with Jack. What we naturally mean 
by this is not simply that the event of Jo dining with Jack 
should occur but that Jo should bring it about, or see to 
it, that this event occurs—that this event should occur 
through Jo’s agency. The study of agency in the context 
of deontic logic began with work by Kanger (1957/1971) 
and was explored in later work by Pérn (1962/1970) and 
Lindahl (1977) on the theory of “normative positions,” 
an attempt to survey the various possible normative 
relations between individuals; this work has now been 
extended and solidified by Sergot (2014). More recently, 
Horty (2001) has explored a deontic logic developed 
within the stit analysis of agency introduced by Belnap, 
Perloff, and Xu (2001). This also has interesting connec- 
tions with game theory, as illustrated, for instance, in 
earlier work by Apostel (1960) and van Benthem (1979) 
as well as more recent work by Tamminga (2013). 

Third, although deontic logic typically focuses on the 
ought-connective introduced above, a connective repre- 
senting permission can be defined as well, as a dual of 
the “ought.” More precisely, where P represents permis- 
sion, the statement PA can be defined as ~O-A—the 
idea is that A is permissible if it is not required that A 
not be the case. This standard treatment of permission 
is sensible and applies naturally in many situations, but 
there are also cases, known as cases of “free choice per- 
mission,” in which this standard treatment seems to give 
the wrong results. Suppose the waiter tells you that, as 
part of your fixed-price meal, you can have either soup 
or salad. If we now let A represent the proposition that 
you have soup and B the proposition that you have salad, 
then it is reasonable to formalize the waiter’s statement 
as P(A v B)—it is permitted that you have soup or you 
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have salad. Of course, we would normally conclude from 
the waiter’s statement that PA , PB—you are permitted to 
have soup and also permitted to have salad. The trouble is 
that, on the standard definition of permission as the dual 
of obligation, P(A v B) does not entail PA , PB—we can- 
not conclude what we want to from what the waiter said. 
The problem of free-choice permission is the problem of 
finding another definition of permission that supports 
the desired inference; a contemporary survey of literature 
on the problem can be found in Hansson (2014). Again, 
this problem has also attracted attention from a game- 
theoretic perspective, as illustrated in work by Dong and 
Roy (2015) and Anglberger, Gratzl, and Roy (2015). 

Fourth, there is the problem of contrary-to-duty 
oughts. To illustrate, return to our example in which, 
as the result of a promise, Jo ought to have dinner with 
Jack. Now suppose the time for dinner comes and goes 
and, for one reason or another, Jo has failed to meet Jack 
for dinner. What then? It seems sensible to conclude 
that Jo should try to make amends to Jack for breaking 
her promise—perhaps she should call up Jack to apolo- 
gize. The trouble is that since standard deontic logic is 
concerned only with ideal situations, in which no obli- 
gations are violated, it has nothing to say about what Jo 
ought to do in the various subideal situations in which 
she has failed to meet some of her obligations. The prob- 
lem of contrary-to-duty oughts is the problem of design- 
ing a deontic logic to deal with these subideal situations, 
telling us what we ought to do once we have violated 
our primary oughts. A very readable introduction to the 
problem, as well as an interesting proposal for a solu- 
tion, is presented by Prakken and Sergot (1996). 

Fifth, and finally, there is the problem of normative 
conflicts, sometimes known as “moral dilemmas.” Let 
us say that a situation gives rise to a normative con- 
flict if it presents each of two conflicting propositions 
as obligatory—if, for example, it supports the truth of 
both OA and OB, where A and B are inconsistent and 
so cannot hold jointly. We often seem to face conflicts 
like this in everyday life, and there are a number of vivid 
examples in philosophy, literature, and law. Perhaps 
the best known of these is Sartre’s (1946) description of 
a student during World War II who felt for reasons of 
patriotism that he ought to leave home to join the Free 
French but who felt also, for reasons of sympathy and 
personal devotion, that he ought to stay at home to care 
for his aged mother. 

Sartre presents this student’s situation in a compelling 
way that does make it seem as if he had been confronted 
with conflicting, and perhaps irreconcilable, moral 
principles. However, if standard deontic logic is correct, 
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Sartre is mistaken: the student did not face a moral con- 
flict—no one ever does, because according to standard 
deontic logic, such a conflict is impossible. This is easy to 
see. For the two statements OA and OB to be supported at 
a world w, both A and B must hold at each world that is 
ideal from the standpoint of w. It is a constraint on stan- 
dard deontic models that the set f(w) of worlds ideal from 
the standpoint of w must be nonempty, so there must 
be some world w’ belonging to this set. Both A and B, 
therefore, must hold at w’, but this is impossible since, by 
assumption, A and B cannot hold jointly. 

This feature of standard deontic logic—that it rules 
out normative conflicts—has received extensive discus- 
sion in the philosophical literature. There is currently 
no consensus among moral theorists on the question 
whether an ideal ethical theory could be structured in 
such a way that moral dilemmas might arise. The issue 
has been addressed, for example, by Donagan (1984), 
Foot (1983), Lemmon (1962), Marcus (1980), and Wil- 
liams (1965). Still, it can seem like an objectionable 
feature of standard deontic logic that it rules out this 
possibility. Because the question is open, and the possi- 
bility of moral dilemmas is a matter for substantive ethi- 
cal discussion, it seems to be inappropriate for a position 
on this issue to be built into the logic of the subject. And 
even if it does turn out, ultimately, that research in eth- 
ics can exclude the possibility of conflicts in a correct 
moral theory, it is useful all the same to have a deontic 
logic that is able to tolerate normative conflicts, since 
they arise in so many other domains. 

The first intuitively adequate account of reasoning 
in the presence of normative conflicts was presented in 
van Fraassen (1973). The logic presented there was inter- 
preted in Reiter’s (1980) default logic by Horty (1994), 
establishing the first clear connection between existing 
deontic and nonmonotonic, or defeasible, logics; this 
interpretation was later developed from a philosophical 
perspective in Horty (2012). The connection between 
deontic and defeasible logics has now been explored in 
some detail. A useful collection of early papers is found 
in Nute (1997). Much of the most interesting recent 
work, summarized by Parent and van der Torre (2014), 
revolves around the framework of input/output logic, 
rather than older frameworks for defeasible logic. The 
most authoritative current treatment of normative con- 
flicts in deontic logic is presented by Goble (2014). 


5. Conclusion 


The field of deontic logic began early in the 20th century 
with the introduction of a number of formal systems, 
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many of them flawed, aimed at analyzing normative 
concepts in philosophy and law. The field was solidified 
toward the middle of the century with the isolation of 
the system now known as standard deontic logic, which 
is sensible and well understood at least from a logical 
perspective. There followed a period of confusion, and 
eventual stagnation, centered on the failure of this stan- 
dard deontic logic to yield what many considered the 
correct results in a variety of puzzle cases, sometimes 
known as “deontic paradoxes.” As a result of an infu- 
sion of new techniques, primarily from computer sci- 
ence, deontic logic is once again active and vibrant, with 
applications not only in computer science but also in a 
number of other fields, such as linguistics, political sci- 
ence, organizational theory, and economics. 

Our goal here has been to present the central ideas 
underlying the system of standard deontic logic and 
to survey some of its more interesting variations. It is 
impossible, in a brief chapter such as this, to cover the 
range of current applications for deontic logic, but just 
to give the reader a sense of these connections, we have 
sketched a few of the relations between deontic logic 
and game theory. 


References 


Anglberger, A. J. J., Gratzl, N., & Roy, O. (2015). Obligation, free 
choice, and the logic of weakest permissions. Review of Symbolic 
Logic, 8(4), 807-827. 


Apostel, L. (1960). Game theory and the interpretation of deon- 
tic logic. Logique et Analyse, 3(10), 70-90. 


Aqvist, L., & Hoepelman, J. (1981). Some theorems about a 
“tree” system of deontic tense logic. In R. Hilpinen (Ed.), New 
studies in deontic logic: Norms, actions, and the foundations of ethics 
(pp. 187-221). Dordrecht, Netherlands: Reidel. 


Belnap, N., Perloff, M., & Xu, M. (2001). Facing the future: 
Agents and choices in our indeterministic world. Oxford, England: 
Oxford University Press. 


Donagan, A. (1984). Consistency in rationalist moral systems. 
Journal of Philosophy, 81, 291-309. 


Dong, H., & Roy, O. (2015). Three deontic logics for rational 
agency in games. Studies in Logic, 8(4), 7-31. 


Foot, P. (1983). Moral realism and moral dilemmas. Journal of 
Philosophy, 80, 379-398. 


Goble, L. (2014). Prima facie norms, normative conflicts, and 
dilemmas. In D. M. Gabbay, J. Horty, X. Parent, R. van der 
Meyden, & L. van der Torre (Eds.), Handbook of deontic logic 
and normative systems (pp. 241-352). London, England: College 
Publications. 


650 


Hansson, S. O. (2014). The varieties of permission. In D. M. 
Gabbay, J. Horty, X. Parent, R. van der Meyden, & L. van der 
Torre (Eds.), Handbook of deontic logic and normative systems 
(pp. 195-240). London, England: College Publications. 


Hilpinen, R., & McNamara, P. (2014). Deontic logic: A historical 
survey and introduction. In D. M. Gabbay, J. Horty, X. Parent, R. 
van der Meyden, & L. van der Torre (Eds.), Handbook of deontic 
logic and normative systems (pp. 3-136). London, England: College 
Publications. 


Horty, J. F. (1994). Moral dilemmas and nonmonotonic logic. 
Journal of Philosophical Logic, 23, 35-65. 


Horty, J. F. (2001). Agency and deontic logic. Oxford, England: 
Oxford University Press. 


Horty, J. F. (2012). Reasons as defaults. Oxford, England: Oxford 
University Press. 


Kanger, S. (1971). New foundations for ethical theory. In R. 
Hilpinen (Ed.), Deontic logic: Introductory and systematic readings 
(pp. 36-58). Dordrecht, Netherlands: Reidel. (Original work 
privately distributed in 1957) 


Lemmon, E. J. (1962). Moral dilemmas. Philosophical Review, 
70, 139-158. 


Lindahl, L. (1977). Position and change: A study in law and logic. 
Dordrecht, Netherlands: Reidel. 


Mally, E. (1926). Grundgesetze des Sollens: Elemente der Logik des 
Willens [Basic laws of ought: Elements of the logic of the will]. 
Graz, Austria: Leuschner & Lubensky. 


Marcus, R. B. (1980). Moral dilemmas and consistency. Journal 
of Philosophy, 77, 121-136. 


Nute, D. (Ed.). (1997). Defeasible deontic logic. Dordrecht, Neth- 
erlands: Kluwer. 


Parent, X., & van der Torre, L. (2014). Input/output logic. 
In D. M. Gabbay, J. Horty, X. Parent, R. van der Meyden, & 
L. van der Torre (Eds.), Handbook of deontic logic and normative 
systems (pp. 409-544). London, England: College Publications. 


Porn, I. (1970). The logic of power (2nd ed.). Oxford, England: 
Blackwell. (Original work published 1962) 


Prakken, H., & Sergot, M. (1996). Contrary-to-duty obligations. 
Studia Logica, 57, 91-115. 


Reiter, R. (1980). A logic for default reasoning. Artificial Intel- 
ligence, 13, 81-132. 


Sartre, J.-P. (1946). L’existentialisme est un humanisme [Existen- 
tialism is a humanism]. Paris, France: Nagel. 


Sergot, M. (2014). Normative positions. In D. M. Gabbay, J. 
Horty, X. Parent, R. van der Meyden, & L. van der Torre (Eds.), 
Handbook of deontic logic and normative systems (pp. 353-406). 
London, England: College Publications. 


John Horty and Olivier Roy 


Tamminga, A. (2013). Deontic logic for strategic games. Erken- 
ntnis, 78, 183-200. 


Thomason, R. (1981). Deontic logic as founded on tense logic. 
In R. Hilpinen (Ed.), New studies in deontic logic: Norms, actions, 
and the foundations of ethics (pp. 165-176). Dordrecht, Nether- 
lands: Reidel. 


Thomason, R. (1984). Combinations of tense and modality. 
In D. M. Gabbay & F. Guenthner (Eds.), Handbook of philosophi- 
cal logic: Vol. II. Extensions of classical logic (pp. 135-165). Dor- 
drecht, Netherlands: Reidel. 


van Benthem, J. (1979). Minimal deontic logics. Bulletin of the 
Section of Logic, 8(1), 36-42. 


van Eck, J. A. (1982). A system of temporally relative modal 
and deontic predicate logic and its philosophical applications 
2. Logique et Analyse, 25, 339-381. 


van Fraassen, B. C. (1973). Values and the heart’s command. 
Journal of Philosophy, 70, 5-19. 


von Wright, G. H. (1951). Deontic logic. Mind, 60, 1-15. 


Whitehead, A. N., & Russell, B. (1910-1913). Principia mathemat- 
ica (3 vols.). Cambridge, England: Cambridge University Press. 


Williams, B. A. O. (1965). Ethical consistency. Proceedings of the 
Aristotelian Society, 39(Suppl.), 103-124. 


11.2 Deontic Reasoning in Psychology 


Shira Elqgayam 


Summary 


Deontic reasoning is the branch of the psychology of 
reasoning that studies how we reason with and about 
normative rules, using logical operators such as “must,” 
“allowed,” and “forbidden.” Thanks to its dual nature, 
combining reasoning with social context, deontic rea- 
soning provides a unique, multilevel view on human 
rationality, bridging between normative and social and 
pragmatic rationality. This chapter reviews some of the 
historical as well as contemporary findings regarding 
deontic thinking, starting with the deontic selection task; 
considers aspects of the normative, social, and pragmatic 
rationality of deontic thinking; explores its cognitive vari- 
ability; and outlines its implications for the great ratio- 
nality debate. The chapter concludes with some thoughts 
about the future of research in deontic reasoning. 


1. Reasoning about Norms and with Norms 


Normative rules surround us wherever we turn: as we 
drive, we conform (hopefully) to traffic laws. We conform 
to norms when we play games, be it chess, Go, or online 
role-playing. Indeed, children as young as two years old 
make spontaneous normative utterances when a player 
deviates from the rules of a game (Rakoczy, Warneken, & 
Tomasello, 2008). We have norms in law, in religion, in 
education, in medical practice. Norms, as Searle (2005) 
aptly put it, are the glue that holds society together. Nor- 
mative rules have their own language; we use terms such 
as must, ought, allowed, and forbidden to express them: 
“T ought to visit my aunt”; “You must support drug rec- 
ommendations with evidence”; “Smoking is forbidden 
anywhere on this train”; “Students are allowed to use a 
pocket calculator at the exam”; and so on. In logic, this 
language is called deontic, from the Greek deon, obliga- 
tion.’ Thus, deontic logic is the logic of rules, norms, and 
regulations (McNamara, 2010; and see chapter 11.1 by 
Horty & Roy, this handbook), and deontic reasoning is the 


branch of the psychology of reasoning that studies how 
we reason with and about normative rules. 

Deontic reasoning provides a unique, multilevel view 
on human rationality, bridging between normative 
rationality, on the one hand, and social and pragmatic 
rationality, on the other: the rationality of conforming 
to normative standards versus the rationality of achiev- 
ing one’s goals, respectively (Evans & Over, 1996). We 
can of course ask how well people perform on deontic 
deductive competence (pretty well), but there is much 
more to deontic reasoning than straightforward deduc- 
tion: deontic reasoning is about norms, firmly linking it 
to social rationality; it also provides the bridge to action, 
so it is perhaps more directly linked to practical ratio- 
nality than most other domains of human inference. 
Moreover, given that deontic reasoning is about norms, 
it occupies a curious, dual place in the great rationality 
debate, with both first-order and second-order implica- 
tions: we can ask whether people reason normatively 
about deontic premises, but we can also explore the type 
of deontic judgments that scientists make when they 
theorize about reasoning and decision making; after all, 
rationality is about should and ought. 

This chapter reviews some of the historical as well as 
contemporary findings regarding deontic thinking, start- 
ing with the deontic selection task; considers aspects of 
normative, social, and pragmatic rationality of deontic 
thinking; explores its cognitive variability; and outlines 
its implications for the great rationality debate. I con- 
clude with some thoughts about the future of research 
in deontic reasoning. 


2. How It All Started: The Deontic Selection Task 


To understand some of the historical context for the 
appearance of deontic reasoning on the stage of the psy- 
chology of reasoning, it helps to keep in mind that for a 
long time, psychologists of reasoning acknowledged one 
type of logic only: the two-valued, extensional, first-order 
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classical logic of Introduction to Logic textbooks. Perhaps 
the most famous (or, if you will, infamous) example of this 
approach is the Wason selection task (Wason, 1960; for 
reviews, see Evans & Over, 2004, chapter 5; Ragni, Kola, & 
Johnson-Laird, 2018). In the abstract version of this fiend- 
ishly difficult task, reasoners are presented with (typically) 
four cards and are told that each card has a letter on one 
side and a number on the other side. Only one side of 
each card is visible, for example, A, N, 8, 4. They are also 
presented with a conditional of the form “If p, then q,” 
for example, “If there is an A on one side of the card, then 
there is an 8 on the other side.” They are told that the 
conditional is a rule that applies to the cards and may 
be true or false. Their task is to pick the cards—and only 
the cards—that need to be turned over in order to test if 
the conditional is true or false. Wason’s normative stan- 
dard was classical logic and Popper’s philosophy of sci- 
ence: with this standard in place, the normatively correct 
choices are the p cards (A in this case) and the not-q cards 
(4 in this case) (although cf. Oaksford & Chater, 1994). In 
this abstract version (technically known as indicative), suc- 
cess rates (in this classical normative sense) are extremely 
low—around 10%, a finding replicated many times since. 

Performance on the abstract task was so dismal that 
psychologists soon started looking for variations of the 
task that produced better success rates. In the famous 
“drinking age” problem (Griggs & Cox, 1982), participants 
are invited to imagine that they are a police officer respon- 
sible for ensuring that the following regulation was fol- 
lowed: “If a person is drinking beer, then the person must 
be over 19.” Four cards represented four people sitting at 
a table and bore the legends “Drinking beer,” “Drinking 
coke,” “22 years of age,” and “16 years of age” (p, not-p, 
q, and not-q, respectively). Selections of p and not-q were a 
staggering 74%—more than seven times as much as in the 
typical abstract version. This facilitation effect is extremely 
robust (Ragni, Kola, & Johnson-Laird, 2018). 

You might have identified the drinking age rule as 
deontic: not only is the term “must” mentioned in the 
conditional itself, but the task also clearly refers to a regu- 
lation, identifying the context as deontic. This is typical: 
deontic versions usually need at least a minimal context 
to trigger facilitation (Evans & Over, 2004). It was not long 
before Cheng and Holyoak (1985) identified this version 
as deontic, thereby introducing the term “deontic” to sci- 
entific discourse for the first time. Their theory of prag- 
matic reasoning schemas proposed that people reason by 
using generalized, yet context-sensitive, knowledge struc- 
tures (the eponymous schemas) defined in terms of classes 
of goals and relationships to these goals. These schemas 
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included the deontic rules of permission and obligation 
as well as the nondeontic schema of causation. 

With the theory of pragmatic reasoning schemas criti- 
cized for offering little explanation beyond labeling the 
effect (Cosmides, 1989; Evans & Over, 2004), a new theory 
emerged: Cosmides and Tooby’s evolutionary theory of 
reasoning (e.g., Cosmides, 1989; Cosmides & Tooby, 1994; 
see also chapter 10.6 by Cosmides & Tooby, this hand- 
book). Reasoning, according to this theory, consists of 
Darwinian algorithms, specialized information-processing 
mechanisms that organize behavior meaningfully and 
adaptively. Deontic reasoning, specifically, was argued to 
be easier because it triggered the Darwinian algorithm of 
social contract, exchange between individuals for mutual 
benefit. Cosmides was the first to identify the important 
role of utility, articulated as costs and benefits, in deontic 
reasoning. In decision theory, “utility” is the technical 
term for things we want to happen (benefits) or want 
not to happen (costs). Cosmides showed that partici- 
pants were quick to identify the p and not-q cards if they 
referred to cheaters—individuals who violated the social 
contract by taking a benefit without paying the cost. 

Pragmatic reasoning schemas and Darwinian algo- 
rithms moved the psychology of deontic reasoning away 
from classical logic, first by identifying it as deontic and, 
second, by highlighting the psychological importance of 
goals and utility. Both theories relied on what came later 
to be known as “content-specific rules.” Manktelow and 
Over (1991), in contrast, made a direct appeal to a more 
general, decision-theoretic approach to utility, rather 
than specialized evolutionary mechanisms. Where util- 
ity lies, they argued, depended on perspective (see also 
Gigerenzer & Hug, 1992)—in this case, social roles. Thus, 
suppose a mother tells her son: 


(a) “If you tidy up, you may go out and play.” 


The utility for the mother is a tidy room, whereas the 
utility for the son is the opportunity to play. Participants 
told to reason from the mother’s perspective tended to 
pick up the not-p & q card (playing without tidying up), 
whereas participants told to reason from the son’s per- 
spective tended to pick up the p & not-q card (that is, 
not being allowed to play even after tidying up). A more 
comprehensive decision-theoretic approach was soon to 
follow: Oaksford and Chater (1994) drew on the con- 
structs of utility and probability to argue that selection 
of cards in the Wason selection task was driven by the 
quest for information gain. Reasoners picked, they pro- 
posed, the cards that provided the most informative 
answers to the question if gq depended on p. These cards 
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differed for the abstract and the deontic version, which 
explained the differences in performance. 

Of course, it is also possible to examine the purely 
logical relations between deontic operators, detached 
from considerations of utility. These, too, demonstrate a 
high degree of conformity to classical logic (Beller, 2008, 
2010). Nevertheless, reasoners also deviate occasionally, 
an effect labeled “illusory inference” in Bucciarelli and 
Johnson-Laird’s (2005) naive deontics theory. 


3. The Social and Pragmatic Rationality 
of Deontic Thinking 


Our use of language to draw inferences and make deci- 
sions is anchored in social context. Under the head- 
ing of “conversational pragmatics,” philosophers have 
highlighted the function of language as a tool of action 
and communication, in a tradition going back to Austin 
(1961), Searle (1969), and Grice (1975). Language has a 
performative function: it does things in the world. When 
we promise a collaborator to finish writing up our share 
of a manuscript by a particular deadline, we create a new 
social reality in which we are under an obligation (Beller, 
Bender, & Kuhnmtinch, 2005). A promise is thus a deontic 
speech act: it acts on the world through language, changing 
it, laying down a new deontic commitment, and people 
are adept at relating deontic speech acts to situations and 
actions in the world (Beller, 2008). In psychology of rea- 
soning, the study of conversational pragmatics goes back 
to Fillenbaum (e.g., 1976), who was the first to identify 
these functions in his work on inducement and advice con- 
ditionals (promises and threats, and tips and warnings, 
respectively). Utility is the telltale sign: deontic speech 
acts create situations in the world where costs and benefits 
attach to actions and their outcomes (Bonnefon, 2009; see 
also chapter 6.4 by Bonnefon, this handbook). 

Given the rich pragmatic nature of inducement and 
advice conditionals, a purely deductive approach is 
unlikely to provide a full psychological understanding 
of their scope and function: psychologists advocated a 
multilevel approach (Beller et al., 2005; Ohm & Thomp- 
son, 2004; Thompson, 2000) encompassing aspects 
such as perlocutionary force (the intended effect), emo- 
tion, linguistic interpretation, and associated utility. For 
example, if the mother in (a) violates her promise and 
does not let the son out to play, the son is likely to feel 
anger (Beller, 2008; Beller et al., 2005). Furthermore, 
promises and threats (aka inducements) create stronger 
obligations than advice conditionals (tips and warnings) 
because they have stronger perlocutionary force: the 
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agent has more control over the consequences (Beller 
et al., 2005; Fillenbaum, 1976). 

The same rich pragmatic nature also makes it diffi- 
cult to disentangle deduction from social pragmatics and 
normative from practical and social rationality—the lat- 
ter Jeak into the former (Bonnefon, 2009). For example, 
the effectiveness and acceptability of deontic speech acts 
are typically associated both with their truth conditions 
and their costs and benefits (Evans, Neilens, Handley, & 
Over, 2008; Evans & Twyman-Musgrove, 1998; Thomp- 
son, Evans, & Handley, 2005; for a review, see chapter 6.4 
by Bonnefon, this handbook). Deontic speech acts also 
evoke a complex net of pragmatic implicatures (linguis- 
tic social inferences licensed by world knowledge). For 
example, the promise in sentence (a) invites the impli- 
cature that if the son does not tidy up, then he will not 
be allowed to go out and play. Indeed, inducement and 
advice conditionals, relative to indicative conditionals, 
tend to inspire a more biconditional interpretation (i.e., 
“Tf and only if p, then q”): participants are more likely 
to draw all four conditional elimination inferences from 
threats and promises (Beller, 2005, 2008; Newstead, Ellis, 
Evans, & Dennis, 1997); but not from the weaker tips and 
warnings (Ohm & Thompson, 2004). This sometimes 
takes the form of a defective biconditional interpretation. 
A full biconditional licenses the inference of both the 
inverse (aka complementary: “If not-p, then not-q”) and the 
converse (aka reversed: “If qg, then p”) forms, respectively. 
However, the inverse inference is more common than the 
converse (Beller et al., 2005; Newstead et al., 1997), some- 
times resulting in what Newstead et al. dubbed as “pattern 
X,” that is, TT and FF cases are judged to be true, TF cases 
are judged false, and FT cases are judged irrelevant. This 
does not extend to conditional inference, though: there 
is no particular preference for Denial of the Antecedent 
(“If p then q; not-p; therefore not-q”) as might have been 
expected given the preference for the inverse implicature. 


4. Deontic Reasoning and Cognitive Variability 


Deontic thinking might be one of the prime candidates 
for a cognitive universal, transcending individual and 
cultural differences. The first line of evidence is cross- 
cultural comparison: deontic speech acts elicit similar 
interpretations at least in two WEIRD* and two non- 
WEIRD cultures, Germany and the United Kingdom, as 
well as China and Tonga, respectively (Song, Bender, & 
Beller, 2009; Sztencel & Clarke, 2018), although there 
was also some variation in responses, mainly based 
on locally accepted behaviors and differing emotional 
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reactions. For example, in a choice task between linguis- 
tic formulations, German participants were more likely 
to choose the explicit threat form in comparison to Ton- 
gan participants, who preferred the indirect speech act 
formulation of promise, while U.K. participants were in 
between. 

An additional line of evidence comes from develop- 
mental studies concerning deontic thinking, particularly 
in early childhood: the normative advantage of deontic 
rules appears in some studies as early as three to four 
years of age (Cummins, 1996a, 1996b; Harris & Nufiez, 
1996; Kanngiesser, K6ymen, & Tomasello, 2017). The 
early development of normativity is of such significance 
that the Journal of Experimental Child Psychology has 
recently published a special issue on the topic (Paulus & 
Schmidt, 2018). The precise age is a matter of debate and 
might be as late as seven years (Dack & Astington, 2011) 
or as early as two years (Rakoczy et al., 2008); neverthe- 
less, the weight of the evidence is that children do not 
need very long to develop adult-like sensitivity to deon- 
tics. For example, like adults (Kilpatrick, Manktelow, & 
Over, 2007), children are sensitive to power relations and 
authority (Chin & Lin, 2018; Dack & Astington, 2011). 

In the study of individual differences, Stanovich and 
West (e.g., 2000) demonstrated a consistent pattern in 
which able, motivated participants tend to do norma- 
tively better on a variety of reasoning and decision- 
making tasks. These results do not always extend to 
performance on the deontic selection task. Here the 
results are more mixed, with some studies finding that 
cognitive ability had no predictive power (e.g., Stano- 
vich & West, 1998) and some studies establishing some 
predictive power (Newstead, Handley, Harley, Wright, & 
Farrelly, 2004). 

With these converging lines of evidence, it would 
seem that deontic thinking has a universal core in 
understanding and interpreting conditionals, which is 
in turn tempered by relative sensitivity to cultural and 
contextual cues. 


5. Generating Deontic Norms: ls-Ought Inference 


Ben wants to cook an Italian dish to impress his girl- 
friend. If he uses a certain brand of Italian olive oil, his 
dish will taste better. Does it then follow that Ben should 
use that brand? Most reasoners agree that he should 
(Elgayam, Thompson, Wilkinson, Evans, & Over, 2015), 
and they also think that someone hearing this would 
be persuaded that he should (Thompson et al., 2005). 
People readily infer the “ought” from the “is,” in effect 
generating brand-new, bespoke deontic rules, where 
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none existed before. Philosophers sometimes refer to 
inferring deontic conclusions from indicative premises 
as the “is-ought fallacy,” or the “is-ought problem” 
(Hudson, 1969; Schurz, 1997).* Hume (1739-1740/2000) 
was the first to identify it—as well as condemn it as 
invalid: if the premises contain no deontic operators, we 
cannot validly draw deontic conclusions. 

But if we cannot generate new norms, how can we 
bridge between goals and actions in a fast-changing 
world? Perhaps more than any other inference, inference 
from is to ought (aka deontic introduction) epitomizes 
the tension between normative rationality, on the one 
hand, and social and practical rationality, on the other 
(Evans & Elqayam, 2020). Normatively speaking, deon- 
tic introduction is deductively invalid, but pragmati- 
cally, we cannot achieve our goals without being able 
to convert goals to norms. We need to be able to infer 
“T ought” from the “I want” of goals and desires, and to 
function in society, we need to be able to convert desires 
to socially binding rules and norms. Deontic introduc- 
tion does all this for us. We also need this inference to 
be flexible, to allow us to act in changing circumstances, 
and it is. Is—ought inference is defeasible: reasoners with- 
draw it when the causal link between action and out- 
come is undermined, or in the presence of conflicting 
norms or conflicting goals. For example, if we know that 
the olive oil is manufactured using exploitative agricul- 
tural practices, we would no longer think that Ben ought 
to use it. 

Elqayam and Evans (2011) argued that psychologists 
sometimes draw on various forms of is-to-ought infer- 
ence to defend their normative stance. For example, 
consider the following question, which is the appropri- 
ate normative solution to the indicative Wason selection 
task: the classic p & not-q selection, or the p-only or p & q 
selections? Stanovich (1999; see also Stanovich & West, 
1998) argued for the former, based on the positive (albeit 
moderate) correlation between these selections and 
cognitive ability and cognitive motivation. Oaksford 
and Chater (2007), in contrast, argued in favor of the 
latter, based on the modal responses to the task. These 
approaches establish the selection among competing 
normative models—which models we should follow—by 
drawing on empirical evidence, hence arguably drawing 
an is—ought inference. Normative rationality in psychol- 
ogy is thus arguably difficult to establish, because there 
are competing normative models, and to support any 
of them with empirical evidence is self-defeating (for 
responses to the debate, cf. Oaksford, 2014; Oaksford 
& Chater, 2011; Stanovich, 2011; and contributions to 
Over & Elqayam, 2016). Moreover, the inference may be 
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pragmatically infelicitous in the context of normative 
rationality—recall that people withdraw deontic conclu- 
sions in the presence of conflicting norms. 


6. Where Do We Go from Here? 


The selection task wars are long behind us, but deontic 
reasoning keeps drawing attention and further research, 
particularly from the viewpoint of social pragmatics (Hil- 
ton, Charalambides, & Hoareau-Blanchet, 2016; Sztencel 
& Clarke, 2018). Much is now integrated into the useful 
framework of the theory of utility conditionals (Bonnefon, 
2009; chapter 6.4 by Bonnefon, this handbook). There are 
still some relatively little-explored areas, however, such 
as cultural differences both in childhood (Miller, Wice, & 
Goyal, 2018) and adulthood. We are also missing studies 
that examine directly the link between moral judgment 
and deontic reasoning, as well as studies that explore fur- 
ther the link to action. Given deontic premises, reasoners 
are happy to draw inferences about other agents’ readi- 
ness to act (Beller, 2005, 2008; Sztencel & Clarke, 2018); 
what we do not know is if endorsement of deontic rules 
also translates to actual action in the world, or at least to 
self-reported readiness to act. Similarly, there is some evi- 
dence that utilitarian moral judgment draws on deontic 
conclusions inferred from indicative premises (Elqayam, 
Wilkinson, Thompson, Over, & Evans, 2017), while deon- 
tological moral judgment correlates with drawing deontic 
conclusions from deontic premises. We need more studies 
looking at relations between deontic reasoning and moral 
judgment. 

Another relatively neglected issue is the variety of 
deontic operators: many studies incorporate just the clas- 
sic permissions and obligations. Very few studies exam- 
ine the middle-strength operators such as “ought” and 
“should not,” although these operators are ubiquitous in 
everyday life. 

Lastly, the recent debates on the scientific rigor of psy- 
chological science are pretty much deontic debates—that 
is to say, normative: they are about the should and the 
ought (and sometimes downright must and must-not) just 
as much as they are about the is. Exploring deontic scien- 
tific thinking could help inform this important debate. 


Notes 


1. Not to be confused with deontological morality, the type of 
moral judgment that relies on preexisting absolute normative 
rules (see section 12 in this handbook for moral judgment). 


2. WEIRD: Western, Educated, Industrialized, Rich, Democratic 
(Henrich, Heine, & Norenzayan, 2010). 
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3. A special case of the is-ought fallacy is the naturalistic fal- 
lacy, where the “ought” is identified as stemming from natural 
properties (Ridge, 2019); see the introductory chapter by Knauff 
and Spohn (this handbook). 
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11.3 Legal Logic 


Eric Hilgendorf 


Summary 


Although the basic method in use for the application of 
law can be characterized as a form of “legal syllogism,” 
the decision-making procedures raise much more diffi- 
cult and complex issues, which are debated in legal the- 
ory and legal methodology. Formal logic plays virtually 
no role here; sophisticated types of logic such as deontic 
logic have so far only found an audience in legal theory. 
Legal logic should therefore not be understood as the 
theory of drawing valid logical conclusions but rather 
as the theory of methods of legal analysis and decision 
making. These decision-making methods exist along 
a continuum, with rule-based decision making at one 
end and the weighing of legal interests at the other end. 
Uncontrolled “weighing” sometimes threatens to super- 
sede ordinary rule-based decision making. 


1. Lawyers and Logic 


Nowadays, legal logic is usually understood as a subject 
in legal theory or legal methodology. It ties in closely 
with conflict resolution in everyday life, by clarifying 
and structuring the forms of argument that can be found 
there.' Without this close connection between legal logic 
and our understanding of everyday life, the law would 
not be able to regulate and resolve social conflicts. Appli- 
cation of the law is always an exercise of power; this 
may explain why the persuasiveness of legal logic is ever 
being tested. Its practical orientation constitutes a cen- 
tral difference between legal logic and the (often highly 
formalized and thus otherworldly) logic that is practiced 
and taught mostly at philosophical faculties.’ 

“Legal logic” can be defined as the sum of all forms of 
argument and rules that lawyers orient themselves toward, 
or at least ought to orient themselves toward, in their legal 
activities (e.g., legal analyses or judicial decision making). 
This proposed definition opens up a wide range of appli- 
cations for legal logic. It is not surprising that terms like 
“judicial logic” (Klug, 1982; Neumann, 2016), “legal logic” 


(Hage, 2005; Joerden, 2010; Weinberger, 1970/1989), 
“normative logic” (Kalinowski, 1972), or “deontic logic” 
(Gabbay, Horty, & Parent, 2013; Joerden, 2017; Navarro 
& Rodriguez, 2014; von Wright, 1977)* are used in a vari- 
ety of different ways today, sometimes employing irrec- 
oncilable definitions. For a very long time, the extreme 
positions between which debates on legal logic were held 
have been, on the one hand, the notion of strictly formal 
logic-based reasoning and legal decision making using the 
“legal syllogism” and, on the other hand, “free judicial 
determination” (freie Rechtsfindung), which exempts deci- 
sion makers from any obligation to comply with rules or 
requirements of law previously conceived. 

The way lawyers think is considered particularly “logi- 
cal”; legal texts are often extolled for their “logic.” Never- 
theless, (deductive) logic, understood as a philosophical 
discipline, hardly plays any role in legal science and law. 
Even “deontic logic” was received only in legal theory 
and only to a limited degree. Together with general prop- 
ositional logic and predicate logic, it has been made the 
subject of some interesting research during the renais- 
sance in legal theory (Hilgendorf, 2005) in the 1960s 
and 1970s (Horovitz, 1972; Kalinowski, 1972; Koch & 
Ri&mann, 1982; Rédig, 1980; Tammelo, 1969; Wagner & 
Haag, 1970; Weinberger, 1974, 1979), without this work, 
however, being able to exert any relevant influence on 
legal practice. It is particularly noteworthy how lawyers, 
including almost all lawyers at university law faculties, 
keep their distance from any form of symbolic or math- 
ematical logic. The same seems to be true for Anglo-Saxon 
countries. The rule judex non calculat still applies. Even a 
theory as committed to logical structures as the Pure The- 
ory of Law by Hans Kelsen can dispense with formal logic 
(Kelsen, 1934/1960; cf. also Kelsen & Klug, 1981). 

However, there are some indications that such reser- 
vation with respect to formal approaches to logic will no 
longer be sustainable in the future. Particularly in view 
of the progress made in the development of artificial 
(machine) intelligence, the question arises as to whether, 
and to what extent, legal norms can be represented in 


660 


such a way that they can be processed by computer 
programs (Ashley, 2017). Such legal algorithms could 
be of great importance in practice. This would be true, 
for example, for automated driving: in the context of 
driving using automated systems, it is being increas- 
ingly discussed how traffic rules could be transformed so 
that they become “computer-readable.” Cars could then 
be programmed with the applicable traffic law, rather 
than relying on the vehicles themselves to detect traffic 
instructions to drivers (e.g., street signs) via optical sen- 
sors, as has been the case up to now. An “algorithmiza- 
tion of the law” would presuppose, however, that legal 
rules could be structured and represented using formal 
logic to a far greater degree than has previously been 
done, so that computers can process them.* 

The task of the law is to regulate conflicts and create 
legal certainty and predictability. Social conflicts should 
be resolved according to rules in an intersubjectively 
understandable manner, and there seems to be a funda- 
mental human need for equal constellations of facts to 
be treated equally. For that reason, legal decision making 
right from the start has something inherently conserva- 
tive about it. The search for consistent solutions to social 
conflicts requires that decision makers have a method- 
ological tool at their disposal to help analyze conflicts, 
identify key issues, and develop solutions that correspond 
to the degree of consistency our sense of justice requires. 
This methodological instrument is “legal logic.”* 

The special reputation lawyers have of being able to 
argue logically is due to several factors: in order to ana- 
lyze and solve a legal case in a systematic, rule-based man- 
ner, one has to ignore many peculiarities of the specific 
situation and instead focus on the abstract (and legally 
relevant) features of the case. This process of abstraction 
distinguishes the methodology of lawyers from unre- 
flected conflict resolution, which seeks to intuitively settle 
individual cases. Legal reasoning consists in concentrat- 
ing on what is legally relevant for coming to a decision, 
ignoring “purely subjective” issues. The lawyer, according 
to a widely held view of good legal practice, should “with- 
draw behind the law.” 

Rules of law used for dealing with social conflicts con- 
sist of a general abstract description of a factual situation 
(e.g., person X killed a human being) and a legal con- 
sequence (e.g., person X should be punished with life 
imprisonment).° The two elements, “factual situation” 
and “legal consequence,” are linked in an “if-then rela- 
tionship”: “If situation X exists, then legal consequence 
Y should occur.” Within the abstractly formulated situ- 
ational description, several individual elements can be 
distinguished, for example, a description of the actor 
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(a person), a description of the target (object) that was 
affected by the actor’s behavior (a human being), and a 
description of the action in question itself (here: killing). 
For legal assessment, only the elements of the offense(s) 
at issue are relevant; all other elements in the descrip- 
tion of the facts are generally ignored as immaterial and 
irrelevant. Not giving consideration to nonmaterial facts 
can be interpreted as the special “objectivity” of legal 
factual analysis. 

The judicial determination of the existence of indi- 
vidual elements of criminal offenses ought to be made in 
accordance with a system of evidence, which is described 
in detail in the rules of procedure, or, in some legal sys- 
tems, in special rules governing the proceedings of the 
court. Violation of the rules of logic, which, in contrast to 
modern formal logic, are mostly understood as the “laws 
of sound thinking,”’ makes a court decision per incuriam 
(“through lack of care”) and is considered a ground for 
appeal. This expresses the special appreciation of logic in 
lawyers’ thinking. However, it is not absolutely clear what 
“logic” is supposed to mean here. What is clear is that it 
includes deductive logic; errors of this kind of logic make 
a judicial decision per incuriam. But the conception of 
“legal logic” proposed in this chapter will be wider than 
the concept of logic employed by appellate courts. 

Astonishingly, the concept of truth, that is, the idea 
of what it means to be “true,” is little discussed in law. 
Lawyers follow, as a matter of course, the correspon- 
dence theory of truth (see, e.g., David, 2002/2016), 
which shows again the close connection between law 
and the concepts and methods of everyday life. More 
recent theories of truth, such as the “consensus theory 
of truth,” have at best found an audience in legal phi- 
losophy and legal theory.® 

Other topics from logic and the theory of science, 
which are also usually discussed in connection with 
theories of argumentation and reasoning, have so far 
received little attention in legal methodology. This 
applies, for example, to the analysis of counterfactual 
reasoning (Evans, 2017, pp. 46ff.),° to the use of heuris- 
tic methods,'° and above all to the study of typical errors 
of reasoning, which include, for example, confound- 
ing causation and correlation (Evans, 2017, pp. 44ff.),"! 
the distortion of rationality standards in argument by 
the inadvertent influence of stereotypes and cognitive 
biases (Evans, 2017, pp. 38ff., 54ff.),'* and the notori- 
ously error-prone use of statistics and probability (Evans, 
2017, pp. 60ff.). 

Any analysis of “legal logic” faces the difficulty that, 
unlike the patterns of argument of natural scientists or 
even mathematicians, patterns of argument of lawyers 


Legal Logic 


differ markedly according to the jurisdiction where 
they were trained. At least two major legal traditions, 
each with characteristic patterns of argument, can be 
identified around the world, namely, the civil law legal 
tradition (Germany, France, Italy, Spain, in a broader 
sense continental Europe, Russia, Central and East Asia 
including Japan, South Korea, and China, as well as 
Latin America), on the one hand, and the common-law 
legal tradition (England and many of its former colonies, 
including most of Canada, India, and the United States), 
on the other hand. 

While positive law predominates in countries follow- 
ing the civil law legal tradition (i.e., statutes, codes, regula- 
tions according to our usual understanding), common-law 
countries rely much more on case law (precedents), also 
referred to as “court jurisprudence.”’? This chapter has 
been written by a lawyer trained in the civil law tradition. 
Learned colleagues trained in common-law jurisdictions 
will probably miss something or other and may also see 
things a bit differently. It needs to be made clear, however, 
that the differences between civil law and common-law 
legal reasoning are in many ways far smaller than is some- 
times thought. 

In the next section, the essential forms and structures 
of legal argumentation and reasoning,“ the observance of 
which is regarded as a necessary condition of legal ratio- 
nality in civil law legal systems, are presented. The Ger- 
man legal system, as one of the core legal systems of the 
civil law legal tradition, has been used for the purposes 
of illustration and comparison. The discussion is supple- 
mented by references to the common-law legal model as 
well as to some subject areas that, in the author’s view, 
have so far often been neglected in the literature on legal 
logic, such as research on “thinking and reasoning,”'* the 
history of legal thinking, and the social and political sig- 
nificance of legal reasoning and argumentation. 


2. The Legal Syllogism 


The rules according to which lawyers deal with social 
conflicts today basically have the structure of “if-then” 
sentences: if X is the case, Y should happen. The first part 
of the sentence is the description of the relevant facts, 
and the second part refers to the legal consequences. In 
civil law jurisdictions, both the relevant facts and the 
legal consequences are fixed in positive law. Since the 
Enlightenment, lawyers and judges have been required 
to follow the written law; this was intended to restrict 
the judicial arbitrariness that was widespread in the early 
modern period, often accompanied by great cruelty, espe- 
cially in the criminal law (Shaffern, 2009, pp. 203ff., 209). 
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The obligation to follow the law is to be ensured via a 
logically derivable relationship: a legal decision must be 
represented in the form of a syllogism with the law as 
its major premise, a description of the facts as its minor 
premise, and a concrete legal decision as its conclusion. 

This concept of application of the law spread in Europe 
as early as the 16th century’; Thomas Hobbes (1588-1679) 
explicitly referred to the principle of the application of a 
general rule to the individual case (Hobbes, 1651/1841, 
pp. 193, 245). In Germany, the model of deductive legal 
analysis (deduktive Rechtsdarstellung) was discussed in the 
context of Christian Wolff’s (1679-1754) rationalism. The 
logical structure of the process of application of the law 
was described in detail by Heinrich K6hler (1685-1737), a 
student of Wolff. Since then, it seems to have become the 
common intellectual property of legal philosophy, legal 
methodology, and practical philosophy. 

The classic German-language treatise on legal logic, 
as it is understood to this day, is by Karl Engisch (1899- 
1990). His Logische Studien zur Gesetzesanwendung (Logi- 
cal Studies on the Application of Law, 1963), first published 
in 1943, is still a standard work on the application of 
the law. According to Engisch, the goal of every appli- 
cation of the law is “to obtain concrete judgments on 
compliance with the law, linked with consequences for 
failure to comply” (Engisch, 1943/1963, p. 3). Engisch 
understood these judgments as being conditional state- 
ments of fact that have a truth value, that is, they can 
be verified as true or false (Sollensurteile). Thus, they are 
not commands (imperatives), which cannot be judged 
to be “true” but are merely “just” or “purposeful” 
statements. 

Engisch’s reconstruction attempt operates completely 
in the context of propositional logic. The logical struc- 
ture of the application of the law can be represented as 
a syllogism: 


If someone kills a human being, he should be punished. 
Person A killed a human being. 


.. Person A should be punished. 


Engisch’s logical model of the application of law has 
been examined and restated several times; among these 
works, the treatise Juristische Logik (Legal Logic) by Ulrich 
Klug (1982) has attained special prominence.'” In the 
1980s, Hans-Joachim Koch and Helmut RtifS{mann sought 
a more precise explanatory model using formal language 
(Koch & RUf mann, 1982, pp. 31ff.). Their explana- 
tions, however, were hardly taken notice of outside legal 
theory. The same applies to prominent non-German 
legal theorists of the time, such as Aulis Aarnio, Carlos 
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Alchourr6n, Eugenio Bulygin, Neil MacCormick, and 
Jerzy Wrdblewski. 


3. Legal Interpretation 


Within the description of the relevant facts of a given 
legal norm, it is possible to further differentiate indi- 
vidual elements, for example (in criminal law), the per- 
petrator, the victim of the action, and the manner in 
which the action (killing, assault, insult, etc.) was per- 
formed. Where law is in written form, the elements are 
sometimes extremely complex. For example, the offense 
of theft in German criminal law is defined as follows: 
“Whosoever takes chattels (i.e., movable property, E.H.) 
belonging to another away from another with the inten- 
tion of unlawfully appropriating them for himself or a 
third person” (§ 242 German Criminal Code [StGB]). 
This legal rule includes the following elements: “prop- 
erty,” “belonging to another,” “movable,” and “takes 
away.” To these objective elements are added the sub- 
jective elements “intention to appropriate” and, as an 
unwritten element, intention (Vorsatz) to steal (i.e., to 
“fulfill” all the objective elements). 

By contrast, in the common-law legal tradition, the 
relevant rules are to a large extent not fixed in writing 
by the legislature but must be sought out by the lawyer 
through the analysis of previous case decisions made by 
judges with the help of “their own sense of fairness, rea- 
sonableness, custom, and good policy” (Schauer, 2012, 
p. 104). Special emphasis is placed on not deviating 
from the fundamental rules established in earlier cases 
(the principle of stare decisis et non quieta movere). Struc- 
turally, therefore, the difference between legal systems 
based on written law and those based on case law is far 
smaller than it might seem at first sight. 

After determining what the individual elements of the 
applicable rule of law are, the next question is whether in 
the present case all elements of the offense are present, 
that is, whether person A killed person B or whether per- 
son A took from person B a movable object that was not 
his own property with the intention of claiming it for 
himself. Most difficulties in conducting the legal analy- 
sis of a practical case are to be found in this second step, 
that is, in the determination of whether the elements of 
the offense are present, rather than in the (ultimately 
trivial) logical structure of the application of the law. 

In civil law systems, the interpretation of the law— 
or, better, the interpretation of the individual linguistic 
elements of the law—generally takes place using four 
methods, typically done in succession, which are usu- 
ally called “canons of interpretation” (Larenz, 1991, 
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pp. 320ff.). The first step is to interpret the words in the 
legal norm in question according to their normal mean- 
ing. The second method is an analysis of the legislative 
history of a statute. The third method, usually referred to 
as “systematic interpretation,” is used when the lawyer 
tries to gain information from the interpretation of other 
legal norms in order to interpret the statute at hand. The 
last method is the interpretation of the law according 
to its “sense and purpose,” also known as “teleological 
interpretation.” The wording of legal texts can usually 
be interpreted in a number of different ways. If the range 
of possible interpretations is not restricted by the his- 
torical or systematic methods of interpretation, then the 
lawyer is faced with the decision of choosing a particular 
interpretation. One rational way to choose would be to 
look at the expected consequences of the decision, to 
evaluate those consequences, and to make a decision in 
the light of that assessment. Legal interpretation thus 
presents itself as a kind of “social technology” (Albert, 
1968/1991, pp. 207ff.)."° 

By means of the methods of legal interpretation, the 
application of law can be structured. These various steps 
and methods of interpretation have heterogeneous meth- 
odological sources. For example, while the interpretation 
of the wording (including its interpretation according to 
legal usage) can be seen as an essentially hermeneutical 
activity, historical interpretation is a historical-empirical 
procedure, namely, the examination of source material 
for information on a particular topic. This, in turn, needs 
to be differentiated from the systematic-interpretation 
approach, which requires a linguistic analysis of the 
relevant law and its associated interpretations in other 
legal statutes, court jurisprudence, and legal doctrine. 
The teleological-interpretation approach contains, on 
the one hand, a significant element of subjective inter- 
pretation; on the other hand, this subjective interpre- 
tation can be structured rationally, for example, on the 
basis of prognoses about the expected effects of certain 
interpretative approaches. In light of these prognoses, it 
is possible to make rational decisions on which interpre- 
tive approaches are preferred. 

The process by which law is applied, as it has devel- 
oped in the most important jurisdictions following the 
civil law legal tradition, does not allow for the law to be 
applied in a purely deductive way. The same is true for 
the Anglo-Saxon legal systems. It certainly does not seem 
possible to algorithmize the process by which the law is 
applied on the basis of current legal rationality and argu- 
mentation standards. All the same, the instruments used 
for the application of the law permit individual elements 
of the application process to be differentiated, the overall 
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process to be structured, and the individual steps in deci- 
sion making to be made transparent and thus criticizable 
and controllable. Therein lie the significant advantages 
of rational (i.e., systematic) legal decision making over 
chance, “gut feelings,” or spontaneous emotions.”” 


4. Weighing of Interests 


“Weighing” plays a special role in legal analysis and legal 
argument today, and its relevance even seems to be grow- 
ing (Klatt, 2013). The concept of “weighing” is nearly 
omnipresent in law and legal science today. At the same 
time, the concept lacks a clear methodological structure; 
what the lawyer does when he “weighs” is therefore often 
unclear (also and especially for the lawyer himself). There- 
fore, from a “rule of law and not of men” perspective, a 
logical analysis of “weighing” is very important in order to 
better understand, predict, and control current practice.”° 

The focus of such a weighing process is the decision 
between two conflicting interests. In many cases, the 
“weighing” concerns rights. “Rights” may be defined as 
“legally protected interests”: Should this chattel belong to 
person A or to person B? When a decision is being made 
on planning permission for a new construction project, 
does the public interest or do private interests prevail? 
Whose interests, those of patient A or those of patient 
B, get priority in a decision on the allocation of medical 
resources? Such questions can be decided by a weighing 
of interests. 

In some cases, weighing of interests is required by 
law. A well-known example of this can be found in § 
34 German Criminal Code (StGB), which states that an 
action is not unlawful in an emergency situation (i.e., 
based on necessity), “if upon weighing the conflicting 
legal interests, in particular the affected legal goods 
(Rechtsgiiter) and the degree of danger facing them, the 
protected legal interest substantially outweighs the one 
interfered with.” So, if a person takes the walking stick 
from a pedestrian in order to defend a child against a 
dog attacking the child, that person does not act unlaw- 
fully because the protected interests (the physical integ- 
rity and possibly even the life of the child) are more 
important (“weigh more”) than the impaired interest 
(the interest of the pedestrian in having the use of his 
walking stick at this moment). 

When a weighing of interests is conducted, however, 
what is compared are not simply the legal interests in 
themselves but also the probabilities with which a viola- 
tion of those legal interests might take place. The par- 
allels to classical decision theory are obvious.”! Thus, a 
situation could exist in which one legal interest would 
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almost certainly be violated, while the infringement of 
a different legal interest appears possible but unlikely. 
An example would be a situation in which a motorist 
swerves to avoid hitting an injured person lying on the 
street, thereby opening up the possibility of hitting a 
pedestrian, injury to whom, however, seems unlikely. In 
such a situation, and taking into account the low risk of 
hitting the pedestrian, there is much that speaks in favor 
of choosing to swerve in order to save the injured person 
on the ground from an (almost) certain death. The result 
of the weighing of interests would therefore be that the 
swerving maneuver should be carried out in preference 
over killing the injured person. 

A particularly difficult category of cases are situations 
in which life must be weighed against life. The debate 
about what to do in such situations can be traced all the 
way back to antiquity. The most famous example show- 
ing this weighing of interests is called the “Plank of Car- 
neades” (Aichele, 2003). After their ship has sunk, two 
shipwrecked seamen are adrift on a wooden plank float- 
ing in the Mediterranean. As the plank becomes sodden 
with seawater, they can foresee that, at some point, it will 
not be able to carry both men any longer. May person 
A push person B into the water to save himself? Or is it 
person B who is allowed to push person A into the water? 
Is there any relevant difference between A and B? Maybe 
it is better that both drown? And what is the logic behind 
such decisions from a more theoretical perspective? 

There are more recent variations of the problem of 
weighing life against life: the cases of cannibalism on 
the high seas in the 19th century (for example, the 
much-discussed English case referred to after the name 
of the ship as the “Mignonette Case””); the hypothetical 
“Switchman’s Problem,” formulated by the German crim- 
inal lawyer Hans Welzel (which the British philosopher 
Philippa Foot transformed into the “Trolley Problem,” 
generating through her writings a wide response in the 
academic literature in English”); and the decision of the 
German Federal Constitutional Court on the Aviation 
Security Act (2006),** where the question was whether 
the police could shoot down a passenger plane that has 
been hijacked by terrorists, causing the deaths of many 
innocent passengers, in order to save the lives of a signif- 
icantly larger number of innocent people on the ground. 

Such “life-against-life” constellations could also be 
resolved in principle on the basis of formulated rules 
of law (e.g., “If in a life-against-life situation only one 
person’s life can be saved, then the person with charac- 
teristic X should be saved”). Apparently, it was already 
difficult for ancient ethicists and lawyers to agree on 
what characteristic X should be. In contrast, a weighing 
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of interests is much more flexible and allows consider- 
ation of a wide variety of factors, such as the specifics of 
the individual case but also the specific preferences of 
the individual who is doing the weighing. 

Solving such weighing-of-interests problems is a con- 
troversial issue. In most civil law countries, a justifica- 
tion for the killing of innocent people has been rejected 
and instead attention has been drawn to the possibility 
of lack of culpability (Entschuldigung) on the part of the 
perpetrators. Such inculpability presupposes, however, 
that the perpetrators themselves were in a desperate 
predicament (i.e., under duress), so that they cannot be 
held personally responsible despite having committed 
an unlawful act. In countries with less developed logical 
(or “structural”) models of legal analysis (lawyers trained 
in continental law refer to this as “legal dogmatics” or 
“legal doctrine”), the distinction between justification and 
culpability is not (yet) made,” and the actor is simply held 
not to have incurred criminal liability, which, however, 
leaves unanswered the question as to whether the action 
as such is considered to be in conformity with the law, that 
is, “lawful,” or whether the judicial system merely refrains 
from holding the actor to be personally blameworthy, that 
is, “culpable” of committing the offense. 

Like hardly any other instrument in the lawyer’s tool- 
kit, the concept of the weighing of interests is based both 
on historically developed practice and on a more or less 
commonly accepted foundation of values and general 
societal preferences. It is obvious that with the plural- 
ization of society, this common ground may become 
increasingly unstable. Weighing of interests is a compet- 
itive alternative to rule-based decision making in con- 
flict situations. This is particularly problematic because 
there is no consensus about the basic rules and criteria 
for weighing interests. The sheer amount of individual 
subjective evaluation is very large. Therefore, the results 
of balancing-of-interests exercises are often extremely 
difficult to predict. In consequence, the often quite 
unreflected weighing of interests is regarded by many— 
and not without reason—as a severe threat to rule-based, 
reviewable, and thus also criticizable, legal decision mak- 
ing (Rtickert, 2011).”° 

Along a hypothetical continuum with rule-based 
legal decision making at one end and legal decision mak- 
ing by means of the weighing of interests at the other, 
there is a wide range of specific forms of legal argumen- 
tation. This includes argumentum e contrario (argument 
from the contrary), argumentum a fortiori (argument from 
the stronger), and argumentum ad absurdum (argument to 
absurdity). None of these forms of legal argument is com- 
pelling from the perspective of logic.” Rather, their value 
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lies in the fact that certain patterns of argumentation that 
are accepted and used not only in law but also in day- 
to-day life can be structured more clearly and thus made 
reviewable. 


5. Analogy 


Analogy, meaning “comparison based on similarity,” 
plays a special role as a legal argumentation method. 
Common lawyers will be reminded here of persuasive 
precedent.** Analogy may well be the most important 
method of legal reasoning. Assume there is a rule of law 
according to which the legal consequence R should occur 
if elements of the offense T; to T,, are present. In a second 
case constellation, the existence of the elements of the 
offense T> to T, is established, but instead of T;, Ty' is pre- 
sent. In such a situation, the question arises as to whether 
the legal consequence R should also occur. If one wishes 
to argue by analogy, this depends on whether T; and Ty’ 
are sufficiently similar to each other. This way of argu- 
mentation seems to be acceptable not only in everyday 
practice but also in most fields of the law, even though 
from a theoretical perspective, the concept of “similar- 
ity” is extremely vague.” 

In criminal law, however, under most legal systems, 
the argument by means of analogy to the detriment of 
the accused is prohibited. In order for criminal liabil- 
ity to be incurred, the exact elements of the offense 
required under the (written) criminal legal provisions 
must be met. This close connection between crimi- 
nal liability and the precise letter of the criminal law 
dates back to the Enlightenment and is intended to 
impose limits on the arbitrary application of the law 
by judges.*° 

Looking at the details, the key elements of compari- 
sons based on similarity are as much in need of clarifi- 
cation as are the structural elements of decisions made 
by balancing: the logic involved in decisions made by 
weighing and by analogy remains obscure.*' One might 
even start with the assumption that the relevant factors 
influencing decisions vary from person to person. Thus, 
the psychology of decision making by weighing of inter- 
ests or by analogy represents a promising future research 
question. A rationalization effect probably does occur, 
however; due to similar patterns of socialization, one 
would expect similar decisions based on the use of weigh- 
ing and analogy. In addition, the key elements of legal 
decision making based on either method must be made 
explicit and thus become reviewable. In this way, the goal 
of subjecting legal decision making to rational criticism 
and control can be achieved. 
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6. Outlook 


What is remarkable about legal logic is its close connec- 
tion to the kinds of reasoning used in everyday life. One 
could regard legal logic as an abstracted structuring, in 
the form of legal rules, of real-life conflict-resolution strat- 
egies. Without this strong link, the law and its champions 
(i.e., lawyers) would not be accepted by society. The spe- 
cial character of legal logic is thus determined by its func- 
tion. Nevertheless, legal logic does not seem arbitrary, 
nor is it merely an expression of specific cultures. Its basic 
elements—the syllogism by which the law is applied to 
the facts and the requirement that the facts be correctly 
established—seem to be valid in all cultures, even if there 
are enormous differences in the ways by which fact is 
ascertained. These structural similarities suggest that the 
different types of legal logic have a common root but 
that its further exploration falls within the domain, not 
of formal logic, but rather of empirical anthropology and 


psychology. 
Notes 


1. For a long time, research into everyday as well as scientific 
reasoning fell within the domain of rhetoric. In the 1970s and 
1980s, a (general) theory of reasoning established itself on this 
basis, from which specifically legal theories of reasoning soon 
developed (cf. Bongiovanni et al., 2018; Feteris, 1999/2017; 
van Eemeren et al., 2014). What is common to almost all of 
these different notions is that deductive logic is considered to 
play a very small role in factual reasoning. 


2. An excellent overview is found in Prakken and Sartor (2015; 
cf. also Hage, 2005). For a historical overview, see Krimphove 
(2017). 


3. For more details, see chapter 11.1 by Horty and Roy (this 
handbook). 


4. Preliminary work on this took place in the early years of 
legal informatics (e.g., R6dig, 1980). An excellent example for 
recent work is Prakken (2017). 


5. On the meaning of “legal logic” as part of the doctrine of 
legal argumentation, see Perelman (1979), Feteris (1999/2017), 
and Bongiovanni et al. (2018). 


6. This is, so to speak, the “standard” for the drafting of legal 
rules (cf. Larenz, 1991, pp. 250ff.; Zippelius, 2012, § 5). 


7. The legal concept of “laws of sound thinking” seems 
extremely naive, considering the now very sophisticated litera- 
ture on “thinking and reasoning.” For a good introduction, see 
Evans (2017). 


8. For a critical evaluation of the “consensus theory of truth,” 
see Rescher (1995, pp. 44ff.). 
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9. For more detail, see chapter 6.1 by Starr (this handbook). 


10. Strictly speaking, the flowcharts for the analysis of cases 
so popular in legal education and legal science could also be 
called “heuristic methods.” For more details on heuristics and 
the law, see Gigerenzer and Engel (2006). 


11. Compare also with section 7 in this handbook. 


12. Compare also with chapter 2.4 by Fiedler, Prager, and 
McCaughey (this handbook). 


13. It should be noted that precedent plays an important role 
both in common and civil law systems (cf. Glendon, Carozza, 
& Picker, 2015, chapter 11/2; Harris, 2016, pp. 190ff., 202ff.). 


14. Compare with details in chapter 11.4 by Prakken, chap- 
ter 5.5 by Hahn and Collins, and chapter 5.6 by Woods (all in 
this handbook). 


15. For a good summary, see Evans (2017); for legal perspec- 
tives, see Spellman and Schauer (2012) and Warner (2005). 


16. See citations in Hilgendorf (1991, pp. 26ff.). 


17. For a more modern and detailed analysis, see chapter 11.4 
by Prakken (this handbook). 


18. For the English translation, see Albert (2014, chapter 28). 


19. Such forms of judicial decision making are often referred 
to as “qadi jurisprudence.” (A “qadi” [or “kadi”] is a judge of a 
traditional sharia court.) 


20. For a detailed analysis of “weighing,” see Modllers 


(2017/2019, pp. 331-364). For an Anglo-Saxon perspective, see 
Broome (2006) and Lord and Maguire (2016). 


21. Compare with chapter 8.2 by Peterson (this handbook). 
22. R. v. Dudley and Stevens (1884) 14 QBD 273 DC. 
23. See Hilgendorf (2018) for more details and references. 


24. Entscheidungen des Bundesverfassungsgerichts, vol. 
pp. 118-166 (judgment of 15.2.2006, Az. 1 BvR 357/05). 


115, 


25. In countries belonging to the common-law legal tradition, 
the distinction between “justification defenses” and “excuse 
defenses” has still not been fully accepted in criminal legal 
theory, although it has been dealt with in more sophisticated 
treatments for decades (cf. Robinson, 1997, pp. 379ff., 477ff.). 


26. Compare also with Ladeur (2004). For the Anglo-Saxon 
perspective on weighing, see Broome (2006) and Lord and 
Maguire (2016). 


27. Klug (1982, pp. 137ff.). 


28. Under the doctrine of binding judicial precedent, a decision 
by a superior court will bind lower courts in the same jurisdiction 
in later cases where the material facts are exactly the same. Some- 
times, however, the material facts are slightly different. Then 
the superior court’s decision is said to be a persuasive precedent 
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rather than a binding precedent, meaning that the lower court 
should consider it but is under no obligation to follow it. 


29. For a thorough analysis, see Goldstone and Son (2012). 


30. It is significant that one of the first legislative measures 
undertaken by the National Socialists after their seizure of power 
in Germany was the lifting of the ban on the use of analogy 
in criminal law. The rule “No punishment without law” was 
replaced with “No offense without punishment.” 


31. For an ambitious attempt to rationalize the concepts of 
“weighing” and “reasoning by analogy” from a logical perspec- 
tive, see Hage (1997). 
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11.4 Logical Models of Legal Argumentation 


Henry Prakken 


Summary 


This article reviews logical models of legal argumentation 
as they are being developed in the field of artificial intel- 
ligence and law. One aim is to show that several aspects 
of legal reasoning that have often been claimed to escape 
formalization can very well be formalized. First the limi- 
tations of logical deduction as a model of legal reasoning 
are discussed. Then models of defeasible legal reasoning 
and legal argumentation, which focus on the generation 
and comparison of reasons or arguments for and against 
legal claims, are discussed. Models of rule-based defeasible 
reasoning address exceptions and rule conflicts. Models 
of legal interpretation emphasize the interplay between 
rules and cases and the role of principles, purposes, and 
values. Models of legal proof account for the uncertainty 
in legal proof in three alternative ways: with Bayesian 
probability theory, with stories, and with argumentation. 
Finally, dialogue models of legal argument address issues 
of procedural rationality and strategic choice. 


1. Logic, Argumentation, and Law 


Models of rationality in the law must be diverse. In a 
legal case, various types of decisions must be made, 
including determining the facts, classifying the facts 
under legal concepts, and deriving legal consequences 
from the classified facts, and each has its own modes of 
reasoning. Therefore, in the law, theoretical and prac- 
tical reasoning must be combined. When determining 
the facts, the modes of reasoning are often probabilistic 
and may involve reasoning about causation and about 
mental attitudes such as intent. Classifying the facts 
under legal concepts involves interpretation. Here the 
prevailing modes of reasoning are analogy, appeals to 
precedent, and the balancing of interests, purposes, and 
values. Finally, when deriving legal consequences from 
the classified facts, the main modes of reasoning are 
deductive but leave room for exceptions to rules and for 
the resolution of rule conflicts. 


While legal reasoning is diverse, still a unified account 
can be given in terms of argumentation. A simple but 
naive model of legal reasoning sees it as logically deduc- 
ing legal consequences from a precisely stated body of 
facts and legal rules. On this account, once a legal text 
and a body of facts have been clearly represented in a 
logical language, the valid inferences are determined by 
the meaning of the representations, and so techniques of 
automated deduction apply. However, this mechanical 
approach leaves out much of what is important in legal 
reasoning. To start with, the law is not just a conceptual 
or axiomatic system but has social objectives and effects, 
which must be taken into account when applying the 
law. For example, in the well-known Riggs v. Palmer case 
in American inheritance law (discussed by Dworkin, 
1977), a grandson had killed his grandfather and then 
claimed his share in the inheritance. The court made an 
exception to inheritance law based on the principle that 
one cannot profit from their own wrongdoing. More- 
over, legislators cannot fully predict in which circum- 
stances the law will have to be applied, so legislation has 
to be formulated in general and abstract terms, such as 
“duty of care” or “misuse of trade secrets,” and qualified 
with general exception categories, such as “self-defense” 
or “unreasonable.” Such concepts and exceptions must 
be interpreted in concrete cases, which creates room for 
disagreement. This is the more so since legal cases often 
involve conflicting interests of opposing parties. The 
prosecution in a criminal case wants the accused con- 
victed while the accused wants to be acquitted, and the 
plaintiff in a civil lawsuit wants to be awarded compen- 
sation for damages while the defendant wants to avoid 
having to pay. These three aspects of the law—the pos- 
sibility that legal rules have unjust outcomes, the tension 
between the general terms of the law and the particulars 
of a case, and the adversarial nature of legal procedures— 
cause legal reasoning to go beyond the meaning of the 
legal rules. It involves appeals to precedent, principle, pol- 
icy, and purpose, as well as the consideration of reasons for 
and against drawing certain conclusions. Therefore, law 
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relies not just on deduction but also on argument.' This 
chapter aims to show that these aspects of legal reasoning, 
which have often been claimed to escape formalization 
(see, e.g., chapter 11.3 by Hilgendorf, this handbook) can 
very well be formalized. 

Accordingly, this chapter” reviews logical models of 
legal argumentation, which have mostly been devel- 
oped in the field of artificial intelligence (AI) and law, 
as formal underpinnings of implemented AI systems for 
supporting legal argumentation and decision making. In 
reviewing the various approaches, we will focus more on 
their underlying ideas and concepts than on their tech- 
nical particulars. First applications of nonmonotonic 
logics for dealing with rule exceptions and rule con- 
flicts are discussed. Then, argumentation-based mod- 
els of legal interpretation that emphasize the interplay 
between rules and cases, as well as the role of principles, 
purposes, and values, are reviewed. As regards fact find- 
ing, argumentation-based, story-based, and probabilistic 
models of legal proof are briefly compared. Finally, dia- 
logical models of legal argument and their use in model- 
ing legal-procedural notions are discussed. 


2. A First Challenge for Deductive Models: 
Exceptions and Rule Conflicts 


Any attempt to logically model reasoning with legal rules 
faces the challenge that legal rules can have exceptions 
and that several conflicting legal rules can apply to a case. 
One problem is that the law often makes legal effects 
dependent on the nonavailability of evidence for excep- 
tions. Consider a legal rule “An offer and an acceptance 
create a binding contract” with an exception in case the 
offeree was insane when accepting the offer: as long as 
there is no evidence that the offeree was insane, the gen- 
eral rule can be applied to defeasibly conclude that there 
is a binding contract. If later evidence is provided that the 
offeree was insane, this conclusion must be retracted. The 
same holds for exceptions that come from sources other 
than statutes, as in the abovementioned Riggs v. Palmer 
case, where a statutory rule was overridden by an unwrit- 
ten principle. The making and retracting of such “defea- 
sible” inferences goes beyond standard deductive models 
of reasoning. Instead, nonmonotonic logics* should be 
used. Such logics allow the making of inferences subject 
to evidence to the contrary and the retraction of such 
inferences when evidence to the contrary comes in. They 
thus formalize one influential idea introduced by Toul- 
min (1958), namely, that “good” arguments can be defea- 
sible in that it is sometimes rational to accept all their 
premises but not their conclusion. 


Henry Prakken 


For representing legal rules and exceptions, two dif- 
ferent logical techniques have been used. The first tech- 
nique adds an additional condition “unless there is an 
exception” to every rule and combines it with a nonprov- 
ability operator, such as logic programming’s negation- 
as-failure (Sergot et al., 1986). Thus, for example, the 
above legal contract rule and its exception can be repre- 
sented as follows (where “~” stands for nonprovability): 


tm: Offer & Acceptance & ~Exception(r) => Binding 
Contract 
t: Insane = Exception(r) 


tz; Insane & ~Exception(r3) = Not BindingContract 


The second technique states a priority between a general 
rule and an exception: 


tm: Offer & Acceptance = BindingContract 
tz: Insane = Not BindingContract 


T, <To2 


Several nonmonotonic logics can also model reason- 
ing about the resolution of rule conflicts. Legal systems 
have general principles for resolving rule conflicts, such 
as preferring the more specific or more recent rule, or 
preferring the rule that is hierarchically superior (e.g., 
“The constitution is superior to statutes”). Moreover, 
specific statutes can have specific conflict resolution 
rules such as “Rules concerning labor contracts over- 
ride rules from general contract law.” If no clear conflict 
resolution principles apply or if these principles them- 
selves conflict, then some approaches allow reasoning 
about which rule preference holds (e.g., Gordon, 1994; 
Prakken & Sartor, 1997; Verheij, Hage, & van den Herik, 
1998). 

Although rule-based nonmonotonic techniques tech- 
nically deviate from deductive logic, their spirit is still 
the same, namely, that of deriving consequences from 
clear and unambiguous representations of legal rules 
and facts. The logical consequences of a representation 
are still clearly defined. While technically, most non- 
monotonic logics allow for alternative conclusion sets 
(see further section 4 below), in legal practice, statutory 
rule-exception structures and legislative hierarchies 
still usually yield unambiguous outcomes. More often, 
conflicts arise not from competing norms but from the 
variety of ways in which they can be interpreted. A real 
challenge for deductive accounts of legal reasoning is 
the gap between the general legal language and the 
particulars of a case. Because of this gap, disagreement 
can arise, and it will arise from the conflicts of interest 
between the parties. 


Logical Models of Legal Argumentation 


3. A Second Challenge for Deductive Models: 
Bridging the Gap between Legal Language 
and the World 


One might think that disagreements about interpretation 
are resolved in concrete cases by courts, so that additional 
rules can be found in case law. If different courts disagree 
on an interpretation, such disagreements could be rep- 
resented with nonmonotonic techniques for handling 
conflicting rules. This was indeed the approach taken by 
Gardner (1987), who designed a program for so-called 
issue spotting in law school contract-law exam prob- 
lems. Given an input case, the task of the program was 
to determine which legal questions involved were easy 
and which were hard, and to solve the easy ones. If all 
the questions were found easy, the program reported the 
case as clear, otherwise as hard. The system contained 
domain knowledge of three different types: legal rules, 
commonsense rules (e.g., on the interpretation of utter- 
ances like “Will you supply ...?”), and rules extracted 
from cases. The program considered a question hard if 
either “the rules run out” or different rules or cases point 
at different solutions, unless it had reasons to prefer one 
solution over the other. For example, it preferred case- 
law rules over conflicting legal or commonsense rules. 

A problem with such approaches is that rules derived 
from precedents are often specific to the case, so a new 
case will rarely exactly match the precedent, and tech- 
niques for handling conflicting rules fall short. Instead, 
reasoning forms are called for in which case-law rules can 
be refined and modified. Here, factors and reasons play 
an important role, and analogies between cases are drawn 
and criticized. Such observations led to a shift in focus 
away from “traditional” nonmonotonic logics and toward 
argumentation-based approaches, in part drawing on gen- 
eral AI models of argumentation such as Dung (1995), Pol- 
lock (1995), and later Modgil and Prakken (2013). Another 
approach is reason-based logic (Hage, 1996; Verheij et al., 
1998), a nonmonotonic logic that models defeasible rea- 
soning as the weighing of reasons for and against a con- 
clusion, where rules are just one source of reasons, along 
with, for example, precedents or principles. 


4. Intermezzo: Logical Models of Argumentation 


Logical models of argumentation as developed in AI view 
defeasible inference as an inherently dialectical matter: an 
argument warrants its conclusion if it is acceptable, and an 
argument is acceptable if, first, it is properly constructed 
and, second, it can be defended against counterarguments. 
They thus formalize the second influential idea introduced 
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by Toulmin (1958), namely, that outside mathematics, the 
validity of arguments does not depend on their syntactic 
form but on whether they can be defended in a rational 
dispute. Argumentation logics define three things: how 
arguments can be constructed, how they can be attacked 
by counterarguments, and how they can be defended 
against such attacks. Argumentation logics are a form of 
nonmonotonic logic, since new information may give rise 
to new counterarguments against arguments that were 
originally acceptable. 

The basis of much logical work reviewed below is 
Dung’s (1995) famous abstract account of argumenta- 
tion, which just assumes a set of arguments (whatever 
they look like) with a binary relation of attack (on what- 
ever grounds). Dung’s theory characterizes various sets S$ 
of arguments that are admissible in that 


1. S is conflict free (i.e., no argument in S attacks an argu- 
ment in S), and 


2. S defends all its members (i.e., all attackers of a mem- 
ber of S are attacked by a member of S). 


Each admissible set is a coherent point of view. Reason- 
ers are usually interested in subset-maximal points of view, 
called extensions. Sometimes a theory has a unique exten- 
sion and sometimes multiple, mutually incompatible 
ones. Consider the two example graphs in figure 11.4.1, 
in which the nodes are arguments and the links are attack 
relations. 

In figure 11.4.1a, there is a unique maximal admis- 
sible set, namely, {A,C} ({C} and the empty set are also 
admissible but they are included in {A,C}). Note that 
{A,B} is not admissible since it is not conflict free, while 
{B} is not admissible since it does not defend B against C. 
So in figure 11.4.1a, both A and C are warranted while B 
is not warranted. 

In figure 11.4.1b, there are two maximal admissible 
sets, namely, {A} and {B}. In the first, A defends itself 
against B while in the second, B defends itself against A. 
Note that {A,B} is not admissible since it is not conflict 
free. So in figure 11.4.1b, neither argument is warranted, 
but both A and B are “defensible” in that they belong to 
alternative coherent points of view. 


. 
G 
Figure 11.4.1 


Two abstract argumentation frameworks. 
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As for the structure of arguments and the nature 
of attack relations, various approaches exist. An early 
example is Prakken and Sartor’s (1997) model of rea- 
soning with legal rules mentioned above in section 2. 
More recent approaches are inspired by the concept of 
argument, or “argumentation,” schemes from argumen- 
tation theory’ (cf. Walton, 1996). Argument schemes 
model stereotypical forms of defeasible, or “presump- 
tive,” reasoning, such as reasoning with defeasible legal 
rules; reasoning with defeasible generalizations; reason- 
ing from expert or witness testimony; various forms of 
analogical, causal, or temporal reasoning; and schemes 
for practical reasoning (in terms of goals fulfilled or 
violated by decision options). Each scheme comes with a 
set of “critical questions,” which point at possible excep- 
tions to the scheme (e.g., Does applying this legal rule 
violate its purpose? Was the witness biased? Are there 
relevant differences between these analogous cases? 
Are there better ways to fulfill the same goal?). In the 
ASPIC+ framework (Modgil & Prakken, 2013), argument 
schemes can be modeled as deductive or defeasible infer- 
ence rules, and applications of such rules can be chained 
into tree-style arguments. All arguments can be attacked 
on their premises while arguments using defeasible rules 
can also be attacked on the use of these rules. Then a 
preference relation on arguments can be defined to see 
which attacks succeed as defeats, after which Dung’s the- 
ory of abstract argumentation can be used with attacks 
replaced by defeats. There are similar approaches that 
are not within ASPIC+or Dung’s theory, such as Car- 
neades (Gordon & Walton, 2009). 


5. Interpreting Legal Concepts with Cases 


Ways of reasoning “when the rules run out” can be found 
in their most explicit form in common-law jurisdictions, 
which traditionally were not statute- but precedent- 
based. This led to reasoning forms where legal rules are 
formulated by courts in the context of particular cases 
and are constantly refined and modified to fit new cir- 
cumstances that were not originally taken into account. 
However, these reasoning forms can also be found in civil 
law jurisdictions, since the concepts in statutory rules 
also have to be interpreted. 


5.1. System-Oriented Work on Legal 

Case-Based Reasoning 

A recent attempt to formalize traditional canons of 
statutory interpretation (see chapter 11.3 by Hilgendorf, 
this handbook) with argument schemes is Walton, Sar- 
tor, and Macagno (2016). A limitation of this work is 
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that interpretation canons are themselves often vague 
and conflicting. Much AI & law work on the interpreta- 
tion of legal concepts instead centers on the notions of 
a dimension, introduced in the HYPO system of Ashley 
(1990), and the related notion of a factor, originating 
from the CATO system of Aleven (2003) (for a detailed 
recent review, see Bench-Capon, 2017). Factors are 
abstractions of fact patterns that favor (“pro-factors”) or 
oppose (“con-factors”) a conclusion. Factors are thus in 
an intermediate position between the specific facts of a 
case and the legal predicates to which such facts may be 
relevant. For example, in CATO, which like HYPO argues 
about misuse of trade secrets, some factors pro misuse 
are that a nondisclosure agreement was signed, that the 
plaintiff had made efforts to maintain secrecy, and that 
the copied product was unique, and some factors con 
misuse are that disclosures were made by the plaintiff 
in negotiations and that the information was reverse- 
engineerable. While in CATO, factors are a basic notion, 
in HYPO, they are defined in terms of dimensions, which 
can have a range of values. Examples of dimensions are 
the number of people to whom a trade secret has been 
disclosed and the severity of the security measures. Dif- 
ferent dimension values can favor different sides and to 
varying degrees, depending on their place in the range 
of values. Factors are then dimension—value pairs that 
favor a particular side to some degree. Generally no clear 
rules can be given on how dimensions and factors relate 
to legal predicates, since each case can have different 
constellations of factors with different degrees of favor- 
ing a side. Hence the importance of past cases. 

HYPO and CATO generate disputes between a plain- 
tiff and a defendant of a legal claim concerning misuse 
of a trade secret. Each move conforms to certain rules 
for analogizing and distinguishing precedents given that 
cases contain sets of factors (in HYPO dimension—value 
pairs) for and against a decision, plus the decision that 
resolves the conflict between the competing factors. A 
precedent is citable for a side if it has the decision wished 
by that side and shares with the current case at least one 
factor that favors that decision. Thus, citable precedents 
do not have to exactly match the current case, which is 
a way of coping with the case-specific nature of case-law 
decisions. A citation can be countered by a counterexam- 
ple, that is, by producing a citable precedent that has the 
opposite outcome. A citation may also be countered by 
distinguishing, that is, by indicating a factor in the cur- 
rent case that is absent in the cited precedent and that 
supports the opposite outcome, or a factor in the pre- 
cedent that is missing in the current case and that sup- 
ports the outcome of the cited case. Dimensions allow 
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an additional way to distinguish a precedent, namely, 
on a shared pro-decision factor that more strongly favors 
the decision in the precedent than in the current case. 

CATO (Aleven, 2003) also has a “factor hierarchy,” 
which expresses expert knowledge about the relations 
between the various factors: more concrete factors 
are reasons for or against the more abstract factors to 
which they are linked. Thus, the factor hierarchy can 
be used to explain why a certain decision was taken, 
which in turn facilitates debate on the relevance of 
differences between cases. For instance, the hierarchy 
positively links the factor security measures taken to the 
more abstract concept efforts to maintain secrecy. Now if 
a precedent contains the first factor but the current case 
lacks it, then the precedent can be distinguished on the 
absence of security measures taken, and this distinction 
can be emphasized by saying that thus no efforts were 
made to maintain secrecy. However, if the current case 
also contains a factor agreed not to disclose information, 
then the factor hierarchy enables downplaying this dis- 
tinction, since it also positively links this factor to efforts 
to maintain secrecy: the party that cited the precedent 
can say that in the current case, just as in the precedent, 
efforts were made to maintain secrecy. 


5.2. Logical Accounts of Legal Case-Based Reasoning 
Various logical accounts of factor-based reasoning with 
cases have been given. A key idea here is that case deci- 
sions give rise to conflicting rules (or conflicting sets of 
reasons) plus a preference rule expressing how the court 
resolved this conflict. In the notation of Prakken and 
Sartor (1998), inspired by Hage (1996) and Loui and Nor- 
man (1995): 


11: Pro-factors = Decision 
tz: Con-factors => Not Decision 


p: ~.. =? Ty, >To 


The priority expresses the court’s decision that the 
pro-factors in the body of rule r, together outweigh 
the con-factors in the body of rule r2. Since the prefer- 
ence is expressed as a rule, arguments can be modeled 
about why one set of factors outweighs another set. This 
approach also allows for “a fortiori” reasoning in that 
adding factors to a pro-decision rule or removing factors 
from a con-decision rule does not affect the rule prior- 
ity. It allows for analogical uses of a pro-decision rule 
by deleting pro factors, which is called “broadening a 
rule.” A broadened rule does not inherit the priority rela- 
tions of the rule from which it is obtained. So if rule r in 
our schematic example is broadened to rule r; by delet- 
ing one of the pro-factors in rule 1, then one cannot 
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conclude from the priority r > r2 in the precedent that rj > 
Tz, Since the deleted factor might have been essential in 
preferring r Over rp. 

Later work has formalized these ideas in an argument- 
scheme approach. For example, Prakken, Wyner, Bench- 
Capon, and Atkinson (2015) model argument schemes 
for deriving preferences between factor sets from cases, 
for citing cases, for distinguishing case citations, and for 
downplaying distinctions within the ASPIC+ framework. 
Uses of these schemes yield a Dung-style argumentation 
framework that can be used for evaluating the arguments. 

Horty (2011) addresses the issue of precedential con- 
straint, that is, under what conditions is a decision in 
a new case allowed or forced by a body of precedents? 
Consider the Keeble case from the common law of prop- 
erty, part of a well-known series of cases on ownership 
of wild animals that are being chased. In Keeble, a pond 
owner placed a duck decoy in his pond with the inten- 
tion to sell the caught ducks for a living. The defendant 
used a gun to scare away the ducks, for no other reason 
than to damage the plaintiff’s business. Here the court 
held for the plaintiff. The pro-plaintiff factors were that 
the plaintiff was hunting for a living (P/Living) and was 
hunting on his own land (OwnLand). The single pro- 
defendant factor was that the animals were not yet 
caught (Not Caught). The issue is whether the plaintiff 
became the owner of the ducks (Owner). In the notation 
of Prakken and Sartor (1998), we have: 


Keeble 
ky: PlLiving, OwnLand = Owner 
ko: Not Caught = Not Owner 


=> k, > ko 


In another precedent, Young, both plaintiff and defen- 
dant were fishermen fishing in the open sea. Just before 
the plaintiff closed his net, the defendant came in and 
caught the fishes with his own net. Here, not only the 
plaintiff but also the defendant were hunting for a living 
(DefLiving). Then we have: 


Young 
yi: PlLiving => Owner 
y2: Not Caught, DefLiving => Not Owner 


To decide Young for the plaintiff, the required priority 
yi > yz cannot be based on the precedent, since y; lacks 
one antecedent of k; and also since y2 adds a con-factor 
to ky. However, deciding Young in accordance with Keeble 
is still allowed by Keeble, since that leaves the decision in 
the precedent unaffected. Horty calls this following the 
precedent. 
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The situation is different if the case base also contains 
a second precedent that is almost like Keeble except that 
the defendant also hunted for a living and in which the 
defendant won: 


Precedent 2 

pi: PlLiving, OwnLand = Owner 

Pz: Not Caught, DefLiving = Not Owner 

= Pp2> Pi 

The priority p2>pi then implies y2>yi, since y2 has 
the same con-decision factors as pz while y; lacks one 
pro-decision factor of p,. But y2 > y; is inconsistent with 
Yi >Yz2, so deciding Young as Keeble is not allowed by the 
extended case base, since it would amount to overruling 
the second precedent. 

In Horty’s approach, not all deviations from a pre- 
cedent are overrulings. Suppose that precedent 2 is not 
in the case base but is a new case, so its decision is not 
yet known. Then, p2>pi is consistent with k, >k:, so 
deciding the new case differently than in the precedent 
is allowed by the case base, since it leaves all decisions 
in precedents unaffected. Horty here says that deciding 
con the original decision in the new case distinguishes the 
precedent. Horty has thus given precise logical formaliza- 
tions of the important common-law notions of follow- 
ing, distinguishing, and overruling a precedent. 

The work described so far does not address why 
factors are pro or con a decision. Berman and Hafner 
(1993) argued that often a factor favors a decision by vir- 
tue of the purposes served or values promoted by taking 
that decision because of the factor. Cases are thus not 
compared on the factors they contain but on the values 
they promote or demote through these factors. Bench- 
Capon and Sartor (2003) computationally modeled this 
approach and illustrated it with the series of cases from 
American property law on ownership of wild animals 
that are being chased. For example, the plaintiff in Keeble 
could argue that people should be protected when pur- 
suing their livelihood, since society benefits from their 
activities. He could also argue that he was hunting on 
his own land, so that the value of protection of property 
is another reason why he should win. The defendant in 
Keeble could argue that since the plaintiff had not yet 
caught the ducks, he had no right to the ducks, since 
if such rights depended on who first saw the animals, 
there would be no clear criterion and the courts would be 
flooded with cases. Thus, the defendant argues that decid- 
ing for him promotes the value of legal certainty. Since 
the plaintiff won in Keeble, we can on this interpretation 
of the case say that the court found that the combination 
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of the values of protecting property and protecting the 
pursuit of livelihood outweighs the single value of legal 
certainty. This value preference can then be cited in new 
cases where the same values are at stake. 

The attention paid to the role of value and purpose led 
to accounts of legal interpretation as a decision problem, 
that is, as a choice between alternative interpretations 
on the basis of the values promoted and demoted by 
these interpretations.° The same account can be given of 
the choice whether to follow a legal rule or to distinguish 
it by formulating a new exception. Philosophers call this 
“practical reasoning.” In AI & law, this line of research has 
especially been pursued by Bench-Capon and Atkinson 
and colleagues (e.g., in Atkinson & Bench-Capon, 2005). 
For example, deciding Young for the defendant can be 
reconstructed as follows. First, practical-reasoning argu- 
ments for following or distinguishing Keeble are stated. 
Then the argument for distinguishing Keeble is preferred 
over the argument for following Keeble, on the ground 
that following Keeble only promotes the value of protect- 
ing livelihood, while distinguishing Keeble in addition 
promotes the value of legal certainty. This account can be 
refined in terms of the relative preferences between goals 
and values, as well as in terms of the extent to which they 
are promoted or demoted. 


6. Establishing the Facts of a Case 


While legal education and scholarship mostly focus on 
reasoning with and about the law, in practice, most cases 
are decided on the facts, so insight as to how facts can be 
proven is crucial for legal practice. AI & law research has 
addressed two main questions: which model of rational 
proof can best be applied to the law, and what is the logi- 
cal nature of important legal evidential constructs like 
burdens of proof and presumptions? We address these 
issues in turn. 


6.1. Models of Legal Proof 

Theoretical models of rational legal proof are generally 
of three kinds: probabilistic, story based, or argument 
based. (In Prakken, Bex, & Mackor [2020], these three 
approaches and combinations of them are illustrated in 
several reconstructions of an actual Dutch murder case.) 
All three approaches acknowledge that evidence can- 
not provide watertight support for a factual claim but 
always leaves room for doubt and uncertainty, but they 
account for this in different ways. Probabilistic approaches 
express uncertainty in terms of numerical probabili- 
ties attached to hypotheses given the evidence. Often a 
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Bayesian approach is taken, nowadays more and more 
with Bayesian networks (Fenton & Neil, 2011).° Probabi- 
listic approaches are by no means uncontroversial. One 
objection is that in legal cases, the required numbers are 
usually not available, either because there are no reli- 
able statistics or because experts or judges are unable or 
reluctant to provide estimates of probabilities. Another 
objection is that probability theory imposes a standard 
of rationality that cannot be attained in practice, so that 
its application would lead to more instead of fewer errors. 
To overcome these limitations of probabilistic models, 
argumentation-based and story-based models have been 
proposed. 

Story-based approaches go back to the work of the psy- 
chologists Bennett and Feldman (1981), who observed 
that the way judges and prosecutors tend to make factual 
judgments is not by probabilistic or logical reasoning 
but by constructing and comparing alternative plausi- 
ble stories about what might have happened. In these 
approaches, the story that best explains the evidence 
must, if it does so to a sufficient degree, be adopted as 
true. These approaches thus model forms of inference to 
the best explanation (Lipton, 1991). 

Both Bayesian and story-based approaches reason from 
hypotheses to the evidence in that, to assess alternative 
hypotheses, they model how likely the evidence is under 
the various hypotheses. In contrast, argumentation-based 
approaches reason from the evidence to the hypothesis, by 
stepwise building evidential arguments from the available 
evidence to the hypotheses. For example, Bex, Prakken, 
Reed, and Walton (2003), inspired by Pollock’s (1995) 
theory of defeasible reasons, model evidential reasoning 
as the application of various argument schemes, such as 
schemes for perception, memory, induction, applying 
generalizations, reasoning with testimonies, and tempo- 
ral persistence. 


6.2 Burdens of Proof and Presumptions 

To deal with uncertainty and defeasibility, legal systems 
use such notions as presumptions and burdens of proof. 
Research has been done on modeling these notions with 
techniques from nonmonotonic logic and argumenta- 
tion (Gordon & Walton, 2009; Governatori & Sartor, 
2010; Prakken & Sartor, 2009). 

Legal presumptions obligate a fact-finder to draw a par- 
ticular inference from a proved fact. Typical examples 
are a presumption that the one who possesses an object 
in good faith is the owner of the object, or a presump- 
tion that when a pedestrian or cyclist is injured in a 
collision with a car, the accident was the driver’s fault. 
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Some presumptions are rebuttable while others are irre- 
buttable. Prakken and Sartor (2009) argue that rebuttable 
presumptions can be logically interpreted as defeasible 
rules in a nonmonotonic logic. They also argue that to 
fully understand notions of burden of proof, logical and 
dialogical models of argumentation must be combined. 
For example, they say that a so-called burden of persua- 
sion for a claim is fulfilled if at the end of a proceeding, 
the claim is acceptable according to the argumentation 
logic applied to the then available evidence. This brings 
us to the dialogical aspects of legal argument. 


7. Legal Reasoning and Legal Procedure 


Legal reasoning usually takes place in the context of a dis- 
pute between adversaries, bound by legal procedures. This 
makes the setting inherently dynamic and multiparty. For 
example, the facts and opinions are not given at the start of 
a case, but the adversaries provide their evidence and argu- 
ments at various stages, and the adjudicator can allocate 
burdens of proof before deciding the dispute. Thus, the 
quality of a legal decision depends not only on its grounds 
but also on how it was reached, which raises issues of pro- 
cedural rationality (cf. Alexy, 1978; Toulmin, 1958). 

AI & law researchers have studied these issues by 
combining defeasible-reasoning models and dialogue 
models of argumentation in formal models of legal- 
procedural notions. Thus, properties of legal procedures 
can be formalized and verified. For example, relevance 
of dialogue moves can be formally characterized, and it 
can be verified whether a procedure always allows the 
adducing of relevant information possessed by a dia- 
logue participant. Insights thus obtained about formal- 
ized procedures can then be used in analyzing or even 
(re)designing actual procedures. 

An influential computational model of a legal pro- 
cedure was Gordon’s (1994) Pleadings Game, which, 
inspired by Alexy (1978), formalized a normative model 
of pleading founded on first principles. It was meant to 
identify the issues to be decided at trial, given what the 
parties had claimed, conceded, challenged, and denied 
in the pleadings phase and what (defeasibly) follows 
from it. Other games define the outcome in terms of 
whether the adversaries in the end agree on the main 
issue or in terms of a decision by an adjudicator. 

The dynamic and multiparty setting of legal procedures 
also raises issues of strategic choice. They are addressed 
in a legal context by Riveret, Prakken, Rotolo, and Sar- 
tor (2008), who apply a combination of game theory 
and argumentation logic to the problem of determining 
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optimal strategies in debates with an adjudicator. In 
such debates, the opposing parties must estimate the 
probability that the premises of their arguments will be 
accepted by the adjudicator. Moreover, they may have 
preferences over the outcome of a debate, so that opti- 
mal strategies are determined by two factors: the prob- 
ability of acceptance of their arguments’ premises by the 
adjudicator and the costs/benefits of such arguments. 


8. Conclusion 


Logical models of legal argument aim to respect that the 
central notion in legal reasoning is not deduction but 
argument. Deduction has its place in legal reasoning, but 
only as part of a larger model of constructing, attacking, 
and comparing arguments. Moreover, logical models of 
legal argument reflect that rules are not the only source of 
legal knowledge: the roles of cases, principles, purposes, 
and values should not be ignored, and these models stress 
the importance of dynamics, procedure, and multiparty 
interaction. While most work has addressed legal inter- 
pretation and normative determinations, some work 
addresses legal proof. Here, too, deduction is just one of 
the tools, as part of probabilistic, argumentation-based, or 
story-based approaches. A main concern here is develop- 
ing models of legal proof that are rationally well founded 
but respect the practical and cognitive constraints faced 
by the participants in court proceedings. Finally, dialogi- 
cal models of legal argument can be used to address issues 
of procedural rationality and strategic choice. 


Notes 


1. See chapter 5.5 by Hahn and Collins and chapter 5.6 by 
Woods (both in this handbook) for introductions to argumen- 
tation theory and Feteris (2017) for an introduction to the 
theory of legal argumentation. 


2. Several parts of this chapter are taken or adapted from 
Bench-Capon, Prakken, and Sartor (2009); Prakken (2015); and 
Prakken and Sartor (2015). Trevor Bench-Capon and Giovanni 
Sartor gave useful feedback on a version of this chapter. 


3. Some ideas concerning nonmonotonic logics are reviewed 
in chapter 5.2 by Rott; chapter 5.3 by Kern-Isberner, Skovgaard- 
Olsen, and Spohn; chapter 5.4 by Gazzo Castaneda and Knauff; 
and chapter 6.1 by Starr (all in this handbook). 


4. See chapter 5.5 by Hahn and Collins and chapter 5.6 by 
Woods (both in this handbook). 


5. For decision theory, see section 8 of this volume. 


6. Bayesian networks are discussed in chapter 4.2 by Hartmann 
(this handbook). 


Henry Prakken 
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Section 12 Moral Thinking and Rationality 


12.1 Rationality and Morality 
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Summary 


Is practical rationality on the side of morality? Is it even 
the benchmark or the ground of morality? Why do what 
morality requires you to do? The diversity of answers 
that are still in the running and of considerations for 
and against them is astonishing. Our aim is to delineate 
the structure of the debate and to locate and clarify some 
major questions, options, and moves. We organize the 
presentation around a pair of prominent sample views, 
linking the rationality of an action to the agent’s desires 
but its morality to the general welfare. We expound how 
different the matter looks for other views of rationality 
or of morality. On the whole, thoughts about each of the 
two normative domains and about conflicting norms 
in general suggest that, even regarding well-informed 
agents, rationality and morality cannot be fully harmo- 
nized. To some extent, convergence will remain gappy 
and contingent. 


1. The Question 


How do the judgments of practical rationality relate 
to those of morality? Let us call that question the RM 
question, with “R” for practical rationality and “M” for 
morality. The RM question is complex because each of 
the two relata is controversial in its own right—what 
is the rational thing to do, and what is the morally 
right thing to do?—and so is the pecking order among 
them: does an action have to be morally right in order 
to qualify as rational, or vice versa, or is there no such 
connection? 

Judgments of the two kinds appear at variance in 
many cases that have existential weight. It may well 
happen, for example, that morality appears to require an 
affluent person to donate eighty percent of her income 
to a charity that saves lives efficiently, whereas practical 
rationality appears to require her to use the same money 
for completing her beloved collection of abstract paint- 
ings. We would expect a satisfactory answer to the RM 


question to get some kind of grip on such constellations. 
The answer should either show that R and M are in con- 
cord after all or tell us, insofar as they are not, what fol- 
lows from the discord for theories of normativity and for 
thoughtful agents. 


2. Actions and Requirements 


As usual, it makes sense to restrict the inquiry. The plan is 
to look at rationality and morality in relation to actions. 
There are other items that beckon for our attention— 
think of the rationality and morality of beliefs, deci- 
sions, desires, emotions, intentions, maxims, or ways of 
life—but we will not extend the discussion to them. One 
consequence of the focus on actions is that the ratio- 
nality that pertains is practical rationality by definition, 
which enables us to omit the adjective “practical” most 
of the time. 

We will restrict the scope further by looking at only one 
kind of assessment by R and M, that of actions as (ratio- 
nally or morally) required, permitted, optional, or forbid- 
den. These are known as the “deontic” assessments, and it 
is the “all things considered” versions of them that we will 
be concerned with. We present the logical relations among 
the deontic terms in figure 12.1.1. There is one couple of 
terms, not in the diagram, that we will reserve for use in 
the domain R: the terms “rational” and “irrational” for 
actions that are rationally permitted and rationally forbid- 
den, respectively. There is another couple, in the diagram, 


permitted, right forbidden, wrong 


required, 
obligatory 
ought to be done 


optional 


Figure 12.1.1 

The main deontic terms and the logical relations among them. 
Terms written in the same field are roughly synonymous. Each 
term can occur in discourse about R and in discourse about M 
(“rationally permitted” vs. “morally permitted,” etc.). 
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that today we will reserve for use in the domain M: in this 
chapter, the terms “right” and “wrong” will be used for 
moral assessments. 


3. Two Sample Doctrines: Instrumentalism 
and Utilitarianism 


It will help to look at the relations between one com- 
mon criterion of R and one common criterion of M and 
to widen the view from there. As to the rationality of 
actions, many criteria that have been proposed are varia- 
tions of one simple tenet: that it is rational for a person 
to try to get what she wants. There are competing ways 
of refining that outlook, and here is the version that will 
serve as our sample of a criterion: An action that the agent 
can perform is rational if and only if the agent believes that 
no other action that she can perform brings about more ful- 
fillment of her intrinsic desires. 

The performing, believing, and desiring in the crite- 
rion should all be understood as happening or obtain- 
ing at the same moment, but the believing and desiring 
don’t have to occur in the agent’s consciousness; they 
may be purely implicit. An intrinsic desire should be 
understood as a desire of something for its own sake, 
not of it as a means to something else; henceforth in 
this chapter, whenever we write “desire,” we will mean 
“intrinsic desire.” The quantitative notion of fulfillment 
takes into account both the number and the strengths 
of desires. 

We will treat the criterion as the defining feature of 
“instrumentalism.”’ The label is apt because the crite- 
rion codifies a view of actions as tools, assessing them 
with respect to their putative efficiency in achieving the 
agent’s ends. Instrumentalism is a close relative of that 
theory of rational decision-making that puts the maxi- 
mization of expected utility (MEU) center stage and plays 
a large role in the behavioral sciences. The MEU theo- 
rist’s probabilities and amounts of utility correspond, by 
and large, to the instrumentalist’s beliefs and amounts of 
desire fulfillment. 

In the moral domain, our sample criterion will be 
utilitarian: An action is morally right if and only if no other 
action that the agent can perform brings about more welfare, 
worldwide. The utilitarian message is that welfare counts, 
no matter whose welfare it is, and that nothing else 
does. What is that all-important stuff called “welfare”? 
One widely held view, and one that we will assume here, 
is that a person’s welfare is the fulfillment of her desires 
and that in consequence, due to conceptual connections 
between pleasure and desire fulfillment, pleasure is an 
important part of welfare.’ 
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4. Convergence 


To what extent do our sample criteria of Rand M move 
in sync? One source of hitches can be an agent's beliefs, 
which play a role in one criterion but not in the other. 
For example, even the most fervent utilitarian can have 
erroneous beliefs about the impact of her actions on the 
amount of general welfare, and those beliefs can make it 
rational for her to do what is morally wrong. 

If we leave aside the threat posed by deficient beliefs, 
we reach more significant notions of convergence and 
divergence. Let us call an agent well-informed if her beliefs 
are in such a good state that the rational thing for her to 
do would not change if we improved them further (that 
is, if we corrected false beliefs or added true ones). Using 
well-informedness as a stepping-stone, let us understand 
“convergence” and “divergence” as follows: R and M con- 
verge insofar as the actions of well-informed agents that 
are morally required are also rational; R and M diverge 
insofar as the opposite is the case. 


4.1 Convergence through Moral Desires 
If instrumentalism is on the right track, the principal 
forces of convergence will have to be desires that point 
in the right direction. Less metaphorically speaking, 
they will have to be desires that, provided the agent is 
well-informed, have a propensity to make it rational for 
her to perform an action that is morally right. Let us call 
them RM desires. Life abounds with such desires, and 
one challenge is to produce a helpful classification. 

Some RM desires aim directly at something that would 
be morally positive in itself. Figure 12.1.2 presents them 
as the “moral desires.” Some of those attitudes even 
“bring up” the topic of morality. Examples are desires 
that the world be a better place, to do the right thing, 
or to be a virtuous person. Other moral desires do not 
invoke morality as such but exhibit the relevant direct- 
ness all the same. Depending on what is morally positive 
in itself, examples might be desires that some specific 
people be happy, that there be a lot of happiness, that 
everybody be treated equally—or desires to keep prom- 
ises and to refrain from telling lies. The propensity that 
defines RM desires is present in either case. Applied to 
our sample moral doctrine, utilitarianism: both if Mary 
desires to do the right thing and if Mary desires to maxi- 
mize welfare worldwide, it holds true that, provided she 
is well-informed, the desire will tend to make it rational 
for her to do the right thing. 

Sympathy and moral sentiment, widespread as they 
are, serve as high-yield sources of those moral desires that 
do not invoke morality.* If Mary sympathizes with others, 


Rationality and Morality 


moral desires (that p) 
p would be mpi 


7% 


invoke moral status 
(e.g., the desire that 
I do the right thing) 


don’t invoke moral 
status (e.g., the desire 
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the desires are often 
due to sympathy or 
moral sentiment 


Figure 12.1.2 
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Some important kinds of RM desires. The abbreviation “mpi” stands for “morally positive in itself.” One and the same desire can 


belong to more than one kind, even on the same level. 


her sympathy is likely to give rise to or even to constitute 
a desire that others fare well. And if Mary has moral sen- 
timents (for example, sentiments of moral admiration, 
indignation, or satisfaction) with regard to acts of a cer- 
tain kind, the sentiments are likely to give rise to or even 
to constitute desires for or against performing acts of that 
kind. The two sources are so rich that there have been 
proposals to feed morality from them alone, giving us an 
“ethics of sympathy” or a “sentimentalist ethics.”* 

Are all moral desires contingent? Is it just as possible 
for a person to have them as to lack them? The ratio- 
nality of morality would be a more robust affair if there 
were numerous strong moral desires that we cannot fail 
to have. An argument has been devised that purports to 
establish the existence of those resources. Everybody, the 
argument aims to establish, necessarily desires that other 
people have pleasure, with the strength of those desires 
proportional to that of the pleasure at issue. The idea 
is that, if you fully represented to yourself that another 
person experiences a specific pleasure, you would (this 
being entailed by full representing) experience that very 
pleasure yourself—and would thus yourself be pleased 
while representing. And since a disposition to be pleased 
when fully representing a state of affairs to oneself is a 
desire that the state of affairs obtain, you desire that the 
other experience the pleasure.* If the argument works, 
those desires—which on conceptual grounds everybody 
holds regarding everybody else all the time—will do a 
sizable part of the work that we are anxious to see done. 


4.2. Convergence and Rationality without Egoism 
Our glance at other-regarding moral desires, no matter 
whether they are necessary or contingent, is a good 


occasion for setting aside egoism. One might think on 
the following grounds that the relations between ratio- 
nality and morality are particularly strained: (i) they 
would be strained if rationality required us to act ego- 
istically, and (ii) according to instrumentalism rational- 
ity requires exactly that, because the word “egoistic” 
stands for the property of acting in the light of one’s 
own desires. 

The train of thought is misguided. The notion of 
“egoism” that is advanced in claim (ii) is both unusual 
and apt to weaken claim (i). The usual understanding is 
that acting on one’s own desires is not a sufficient con- 
dition for being egoistic but that it also matters what 
those desires are. The usual understanding is that, if a 
good-hearted person strongly desires that other people 
fare well and acts on such desires, she is not an egoist 
but an altruist, while an egoist is a person who lacks or 
fails to put into action such desires. To be sure, some- 
body might go along with that understanding itself but 
still link instrumentalism to egoism by adding the claim 
that such desires do not exist. However, we know of no 
sound argument for the additional claim.° 

The misclassification of rationality as egoistic can also 
result from sloppy thinking about the combination of 
instrumentalism and the theory of welfare as desire ful- 
fillment. It is true that the combination entails some con- 
nection between rational action and the agent’s welfare: 
in acting rationally at point of time t, the well-informed 
person-stage Mary-at-point-of-time-t maximizes the ful- 
fillment of its desires and cannot help thereby maxi- 
mizing its own welfare. We should beware of letting 
that connection blur the picture. In the first place, the 
rational person (or person-stage) at issue need not be 
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concerned with her (or its) own welfare and can act, due 
to the nature of her (or its) desires, highly altruistically. 
Secondly, the person-stage that acts can still fail to maxi- 
mize the welfare of the entire person, who is extended 
over time and may have different desires later. Thirdly, 
even the limited connection we are looking at requires 
as one ingredient a certain view of welfare; if we adopt a 
hedonistic view instead (seeing the welfare of an entity as 
the pleasure that the entity has and not as the fulfillment 
of her desires) and keep instrumentalism, the connection 
vanishes. 

According to most conceptions of rationality, instru- 
mentalism included, rationality does not require us to 
act egoistically. We need to distinguish the question 
how rationality relates to morality from the questions 
how egoism, how prudence, and how an agent’s self- 
interest, self-love, or welfare relate to morality.’ Many 
well-known discussions from the history of philosophy 
are largely about questions of the second kind. Plato, for 
example, assigns a key role in the Republic to the case of 
Gyges, who is ruthless in using for his own advantage 
his power to become invisible; Aristotle writes about 
virtue as a constituent of an agent’s flourishing in the 
Nicomachean Ethics and Henry Sidgwick about individ- 
ual versus universal happiness in The Methods of Ethics. 
Those discussions apply to our question at best partly or 
indirectly. 


4.3, Convergence through Morally Causal Desires 
Some RM desires, also indicated in figure 12.1.2, are 
related to the good or the right in a different way. If a 
person desires p and p would be causally linked to some- 
thing that is morally positive in itself, we will speak 
of a “morally causal desire.” We distinguish subgroups 
of such desires by distinguishing three kinds of the 
linkedness. 

A desire that p is in the first subgroup if p would 
cause something morally positive. For example, a per- 
son desires to cultivate a garden, to keep the kitchen 
clean, or to mend broken bones, and her doing these 
things would cause pleasure in the beholders, eaters, and 
patients and would thus cause something that is mor- 
ally positive in itself. A desire that p is in the second 
subgroup if, conversely, something that is morally posi- 
tive in itself would cause p. For example, Mary desires 
that her parents treat her to cake, which the parents do 
only when, and in that case because, they are happy. 
The treat is desired and is the effect of the happiness, 
with the happiness being morally positive in itself. A 
desire that p is in the third subgroup if p and something 
that is morally positive in itself have a common cause. 
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Consider, for example, heartless Mary, who does not 
care about the victims of malaria but who desires to be 
praised at a charity ball for donating to the fight against 
malaria. If Mary donates, neither does the praise cause 
the thing that is morally positive in itself (the praise 
does not causally affect the saving of the lives) nor vice 
versa, but the two have a common cause: Mary’s dona- 
tion. In such cases, too, the one comes with the other, 
and that matters for RM purposes. 

The key feature, wherever morally causal desires are 
in play, is the indirectness. A desire can fail to be directed 
at anything that would be morally positive in itself and 
still tend to make actions rational that are (or that cause 
things that are) morally positive in themselves. Links of 
the relevant kind are legion: you do the right thing and 
some “other” desire of yours is fulfilled. In that sense, 
large chunks of morality are connected to rationality 
through carrots and sticks. 

The incentives can be external—think of fellow- 
citizens who in response to your morally positive behav- 
ior honor and help you and refrain from ignoring, 
deriding, jailing, or lynching you. The incentives can 
also be internal—think of the joys of believing to have 
done what was good or right and of the absence of pangs 
and remorse. Even when the incentives are internal, the 
constellation differs from the one that characterizes the 
first main group of RM desires, the moral desires. Two 
distinctions apply. In the first place, when you desire 
your joy of doing good and when you desire to do good, 
those are two different desires.* Secondly, we should in 
some cases distinguish even for one and the same desire 
between the reason to put it into one group and the rea- 
son to put it into another. Consider, for example, your 
desiring your own joy of having done good. The reason 
to count that desire as a morally causal one (your joy is 
caused by something morally positive) is different from 
the reason to count it as a moral one (your joy, too, is 
something morally positive in itself). One desire can be 
both. 

Incentives are studied by game theorists in partic- 
ular. We understand quite well by now, with regard to 
several moral standards, how even agents who have no 
moral desires and are at the same time without ifs and 
buts instrumentally rational find themselves drawn into 
actions and outcomes that are morally positive. Under 
various conditions, repeated encounters in the same 
group of such agents become, not least because rewards 
and punishments can emerge, a breeding ground for 
various amounts of cooperation, equality, justice, soli- 
darity, trust, and public good. While significant general 
results, most prominently a family known as the “folk 
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theorems” of game theory, have been established by 
mathematical methods of the more traditional kind, 
there is also evidence from agent-based computer simu- 
lations, sometimes involving entire artificial societies: 
the rational thing to do for the virtual agents, it turns 
out, is often to spare or even to assist each other.” 

So much for the many devices in human minds and 
societies that make it rational for people to do the right 
thing a lot of the time. On the other hand, given criteria 
of R or M resembling our two samples, there is not much 
hope of full convergence. In the example from our open- 
ing paragraphs, the agent might favor abstract paintings 
over human lives, which in the absence of competing 
considerations would make it the rational thing for her 
to violate her moral obligation. When incentives and 
the agent’s moral desires do not add up, rationality will 
take a stand against morality. 


5. Resisting Divergence by Aligning Rationality 
with Morality 


The dominant impetus among philosophers is to keep 
the divergence of R and M in check. A proof that there is 
no divergence is the Holy Grail of practical philosophy, 
yearned for but hard to attain. Some philosophers go as 
far as to let their thinking on R itself or on M itself be gov- 
erned by the premiss of “moral rationalism”—the claim 
that every action that is morally required is rational.'° 
With that sweeping premiss or without, answers to ques- 
tions like “Why be moral?” or “Why act morally?” are 
sought after, and the risk that, even with regard to well- 
informed persons, sometimes no good answer emerges is 
often perceived as an invitation to rethink R or M. Those 
who want to rethink in order to reduce or even eliminate 
divergence have essentially two options: they assimilate 
rationality to morality or vice versa. They could also com- 
bine the two moves, moralizing rationality and rational- 
izing morality, but for the bigger picture it will suffice if 
we treat each of the two in turn. 


5.1. Smaller Departures from Instrumentalism 

As to alternative views of practical rationality, modest 
deviations from instrumentalism make some difference 
regarding divergences. One example is the temporal 
extension of the conative basis. We could modify the 
instrumentalist criterion so that it covers not just the 
desires that the agent has at the time of acting but all 
those that she had, has, or will have. The modification 
would prevent the rational agent from behaving ruth- 
lessly toward her past self or her future self. Morally 
speaking, it would be a step in the right direction—but 
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only a small, intrapersonal one. Anchoring in a rational 
agent regard for the welfare of all her own person-stages 
is a far cry from anchoring in that agent regard for every- 
body’s welfare. 

A second example originates in the observation that, 
if instrumentalism is correct, rational agents can get 
caught in traps of practical rationality. Such traps are sit- 
uations in which, if every agent acts rationally, each of 
them receives, predictably, less fulfillment of her desires 
than she would if everyone acted irrationally. General 
compliance with the requirements of rationality thus 
leads to the opposite of the moral goal, which is general 
welfare. 

Many real-life situations appear to be traps of prac- 
tical rationality. Consider societies that become much 
nastier because people carry weapons. Each individual 
reasons that to carry a weapon is likely to be more con- 
ducive to the fulfillment of her desires either way—that 
is, both if people whom she encounters carry a weapon 
and if they don’t. Because of the individual decisions 
for weapons, the considerable advantages of a weapon- 
less society and of avoiding an arms race remain out of 
reach for all. Even if a weaponless society could still be 
achieved by changing the individual decisions through 
the imposing of sanctions, the sanctioning itself would 
gobble resources and thus welfare. The ubiquity of such 
traps in human interaction and the unequivocalness of 
the moral setbacks—less welfare for one and all—have 
incited a vast complex of research in ethics, economics, 
and psychology.'' 

Can we devise a criterion of rationality that preserves 
the spirit of instrumentalism but spares us the traps? 
David Gauthier is one of the theoreticians who have 
tried.” Gauthier suggests, controversially, that a rational 
agent can choose, “on utility-maximizing grounds, not 
to make further choices on those grounds,” but to adopt 
a certain stance that will determine her behavior. She 
will then be a “constrained maximizer.” The stance that 
Gauthier has in mind is roughly this: I will do, provided 
that so will the others who are involved, my part in 
making possible an outcome that is better for everybody 
than the outcome that unconstrained maximizers could 
achieve. Since an agent who has recognizably adopted 
such a stance will sometimes produce and reap fruits of 
cooperation that are not available to one who hasn't, it 
is rational to become such an agent. Individual instru- 
mental rationality is declared to be a tad more collective 
than we thought. 

If Gauthier’s claims are sound and help us make head- 
way with the RM question, they do so within limits. 
Gauthier’s ambition is restricted to establishing rational 
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support for a morality of mutual benefiting. He has no 
ambition to provide support for moral obligations toward 
beings who are in need but have nothing to offer. 


5.2 Larger Departures from Instrumentalism 

We find more radical consequences for the shape and 
extent of divergence when we turn our attention to 
more radically different pictures of practical rationality. 
The main move is the dethroning of desire. 

Suppose that we break with instrumentalism by replac- 
ing the appeal to desire fulfillment with an appeal to 
conformity to reasons. Let us agree that facts can be rea- 
sons and that constellations of facts (for example, regard- 
ing pains and cures) can make it the case that there is, 
altogether, for a person more reason to perform one 
action (say, to see her cardiologist) than another (say, 
to see her homeopath). The new criterion of rationality 
could then read as follows: An action that the agent can 
perform is rational if and only if she believes that there is not 
more reason for her to perform another action instead.'* Sup- 
pose further that we take many reasons to be “worldly” 
in that they do not involve the agent’s desires. Perhaps, 
for example, that Peter’s education would be finished if he 
received a donation is a reason to donate, even if the agent 
desires no such finishing or donating and desires noth- 
ing that comes with it; or perhaps that serenading the 
moon would increase the glory of the moon is a reason to 
serenade the moon.” 

The impact on the RM nexus would be momentous. 
The task of showing that it is rational for a well-informed 
agent to do what morally she ought to do no longer has 
as its central component the task to track down a fitting 
constellation of desires of hers but the task to identify a 
fitting constellation of reasons for her to do things. In 
order for the entire scheme to be successful, worldly rea- 
sons for actions need to exist and to have a considerable 
affinity to morality and to be connected to rationality as 
stated by the new criterion. Whether all or at least some 
of the three conditions are fulfilled is controversial. 

Which morality you take to be connectible to 
rationality-in-the-new-spirit will depend on your views, 
possibly your intuitions, about the realm of reasons. If 
morality is concerned with promise-keeping, loyalty, or 
the increase of human knowledge, and so are reasons to 
do things and in the same proportion, then every well- 
informed rational agent will do the right thing: keep her 
promises and so on. Our sample moral doctrine, utilitar- 
ianism, is no exception. If there is always most reason to 
maximize the amount of the fulfillment of everybody’s 
desires, then every well-informed rational agent will do 
the right thing and maximize that amount. 
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Following a markedly different path, Immanuel Kant, 
too, ends up advocating some variety of rational benev- 
olence, maximization included. We read that a rational 
being would try, “as far as he can, to advance the ends 
of others” and would come to the conclusion: “the ends 
of a subject that is an end in itself must, as much as 
possible, also be my ends.”!° In which way according to 
Kant rationality secures so much morality is extraordi- 
narily contentious, even among his followers. He draws 
on the claim that a rational being is autonomous in the 
literal sense of giving herself a law. To Kant, that feature 
appears imbued with moral significance. Being autono- 
mous, the agent’s rational will is not pushed around by 
anything, not even by the agent’s own inclinations; it 
finds itself with nothing left to be constituted by than 
the respect for rationality itself and for people who have 
it and for their ends; and, such being the character of 
laws, that will is general, not concerned with one person 
or group in particular. Sound statements of those con- 
nections remain a desideratum. 

As expected, the picture of the relation between R and 
M changes when that of R changes. Our brief encounters 
with the “worldly reasons, not desires” approach and 
the “autonomy, not desires” approach have illustrated 
tectonic movements and the hopes of convergence that 
can be associated with them. 


6. Resisting Divergence by Aligning Morality 
with Rationality 


If divergence is to be avoided, what about getting to 
work at the other end? We could truncate morality. The 
fewer actions are morally obligatory in the first place, 
the smaller the risk that an action is morally obligatory 
but irrational. 

The demands of some moralities are so removed from 
most agents’ conative constitution that cutting down on 
the demanding looks particularly promising. According 
to utilitarianism, for example, an agent ought to give 
the same weight to her own welfare and to that of her 
nearest and dearest as to everybody else’s. Since hardly 
an agent’s desires manifest such impartiality, utilitarian 
obligations that it is irrational for the obligated agent to 
comply with are thick on the ground. 

Various moral prerogatives for agents have been sug- 
gested. We could permit the agent, for some factor k > 1, 
to attach up to k times as much weight in her decisions 
and actions to her own welfare as to the welfare of every- 
body else; or permit her, for some / > 0, never to give up 
more than / units of her own welfare; or permit her, for 
some threshold value m, never to make her own welfare 
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fall below m.’° We could think of other permissions, too, 
concerning her projects rather than her welfare or con- 
cerning the welfare of those she is close to rather than 
just her own. Every such prerogative is a loosening of 
utilitarianism; it would allow the agent to be partial in 
the sense of granting in her decisions extra force to this 
or that, even if the amount of worldwide welfare suffers. 

Prerogatives are apt to narrow the gap between R and 
M but will not close it. A morality that involves a pre- 
rogative will still demand something (for instance, that 
the interests of others be respected to some extent), and 
there is bound to be some well-informed agent who in 
some situation has desires that make it irrational for her 
to comply even with those moderate demands. 

Structurally, contract theories in the Hobbesian tra- 
dition inhabit the same middle ground. Those theories 
are reciprocitarian. They say, with various qualifications, 
that what a person morally ought to do in relation to 
another person is to play by rules that satisfy the follow- 
ing condition: the fulfillment of the desires of each of 
the persons would increase if each of them, rather than 
none, played by those rules. For example, if both refrain 
from insulting each other, that will save both of them 
some distress. 

Once again, we find the curtailing of moral obligation. 
If contractarians are right, there are fewer moral obliga- 
tions than we thought and thus fewer that it might be 
irrational to meet. Most notably, about all persons (and 
other beings) who are not in a position to increase the 
fulfillment of her desires, the contractarian will say that 
she owes them nothing. And once again we also find 
that the curtailing does not suffice to provide full align- 
ment with rationality. Sometimes a well-informed agent 
will have and see the possibility of maximizing the ful- 
fillment of her desires by breaking even the few rules of 
that “minimal morality” and getting away with it. And 
no earthly regime of sanctions will eliminate such pos- 
sibilities, since all such regimes are gappy.”’ 

Some thinkers have suggested that we go one step 
further and prevent all divergence by fully rationalizing 
morality. The proposal is to combine moral rationalism 
with the claim that instrumentalism is by and large on 
the right track. Gilbert Harman endorses the combina- 
tion. He writes: “If S says that (morally) A ought to do 
D, S implies that A has reasons to do D.” And he contin- 
ues: “I assume that the possession of rationality is not 
sufficient to provide a source for relevant reasons, that 
certain desires, goals, or intentions are also necessary.” 
Since agents might lack the relevant attitudes, Harman 
infers “that there might be no reasons at all for a being 
from outer space to avoid harm to us” and “that, for 


687 


Hitler, there might have been no reason at all not to 
order the extermination of the Jews.”'® 

The moral consequences are remarkable. If the moral 
“ought” requires reasons, and reasons require desires, 
but the desires aren’t in place and thus neither are the 
reasons, then neither is the moral “ought.” While Har- 
man invites us to say other things about Hitler (for exam- 
ple, that Hitler is evil or our enemy), he claims that the 
moral “ought” is out of place. It is “odd to say,” Harman 
asserts, that “it was wrong of Hitler to have acted as he 
did” or that “Hitler ought morally not to have ordered 
the extermination of the Jews.” 

And so the rationalization of the moral “ought” would 
be completed. The approach is Procrustean. There is no 
mismatch between the moral “ought” and the practical 
rationality of its addressees because every instance of a 
moral “ought” that would not conform is discarded. 


7. Accepting Divergence 


Should we accept that R and M diverge? We should if it 
seems to us that they do and we see no reason to resist 
the claim. The two parts of that condition deserve sepa- 
rate treatments. 


7.1. Finding Some Divergence in the First Place 

Not many of us enter the inquiry holding fully con- 
vergent views of R and of M, each of which they deem 
plausible in its own right, independently of any pressure 
to see the two in line. It is true that there is full conver- 
gence according to some views of R and of M that we 
have encountered, but the independent plausibility of 
those views is the crux. 

On the rational side, how plausible is the claim that 
something other than the agent’s aims and projects 
rules the roost? The subjective picture exerts a consid- 
erable pull: if you’ve set your heart on something, it is 
rational for you to go after it, and if you haven’t, it is 
not. On the moral side, the questions are inverse. How 
plausible is the claim that, provided you are indifferent 
to other people’s welfare and have to fear no backlash 
from wrecking it, you are morally permitted to wreck it? 
On either side, the claims that would secure full conver- 
gence do not have the ring of truth. 

The problem does not just arise when, at least regard- 
ing well-informed agents, one of the two domains is 
claimed to fully look the way we always took the other 
one to look, with the implausibility due to the fact that 
one view does all the reaching out. If a view of R and a 
view of M met halfway, the implausibility would be dis- 
tributed evenly but not lessened. Although we pointed 
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out that some independently plausible components 
with a conciliatory effect may well be missing from our 
two sample doctrines (which we didn’t call sample doc- 
trines for nothing), no such components are in sight 
that would happen to dovetail, resulting in a perfect fit. 


7.2 Not Theorizing Away the Divergence We Find 
Matters would look different if our thinking were driven 
by the will to rule out divergence. Maybe there are gen- 
eral considerations that speak against divergence and 
that should be taken into account when we form our 
views of R and M? It is surprisingly difficult to find dis- 
tinct statements of such considerations in the literature, 
but here is a brief attempt to articulate and assess some 
candidates. 

The argument from the negativity of non-compliance 
says: “Since people by and large act rationally, every mor- 
ally required act that is irrational is a morally required 
act that is unlikely to be performed—which is a bad 
thing.” We respond that any such badness would be a 
sad truth and that good theorizing should acknowledge 
truths, sad ones included. The badness at issue does not 
provide a right kind of reason for changing our views of 
Ror M. 

A follow-up argument adduces the pointlessness of 
moral judgment due to the negativity of non-compliance: 
“The point of engaging in moral judgment is to avoid 
the said negativity. Why bother if the project does not 
boost the right and the good through compliance?” Part 
of the answer is that not only are there many different 
functions that moral thinking, judging, and speaking 
have, but also many different paths, including indirect 
ones, on which the function of boosting the good and 
the right can be fulfilled. Some of the moral point of 
considering or making or uttering the moral judgment 
that, say, Mary ought to spend half of her spare time 
fighting for a certain cause may well be independent of 
the factual question whether Mary ends up conforming 
to the judgment. Sorting out and signaling our moral 
view of the matter can help shaping decisions, educa- 
tion, outlooks, politics, relationships, and sanctions in 
a myriad of ways, many of which do not even relate to 
Mary in particular. 

Next, there is the threat of the unjustifiedness of moral 
judgments: “To ‘provide morality with a foundation’ or to 
‘justify moral judgments’ is or includes showing that it 
would be rational to act, if the opportunity arose, in line 
with the moral judgments at issue. Moral judgments that 
are subject to divergences are therefore unfounded and 
unjustified.” The complaint is worth pondering. Still, 
you justify something to somebody, and so the premiss 
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of the complaint licenses at best the conclusion that, in 
cases of divergence, the “ought” judgment has not been 
justified to all persons who, according to the judgment, 
ought to do a certain thing under certain circumstances. 
But maybe to some of them. Secondly, the premiss of 
the complaint is controversial in that there are other 
conceptions of what it is to justify a moral judgment. 
Thirdly, let’s not forget that generally speaking, since 
justification stops somewhere, the use of unjustified 
items might be respectable. 

A final argument asserts the inacceptability of norma- 
tive impasses: “When an agent grasps the fact of the 
situation and the morally required action is irrational, 
what is there left for us to tell her? We can tell her that 
two kinds of norms that apply to her impending action 
point in opposite directions, irreconcilably so, and that 
we have no third kind of norm that would adjudicate 
between them. We can wish her good luck at the norma- 
tive crossroads and move on. None of that is of any help 
to her, and theorizing about normativity should do bet- 
ter than that. It should avoid divergences.” 

The most general reply to the objection is that theo- 
rizing about normativity should “do better” only if there 
is independent evidence that it got the lay of the land 
wrong. That evidence would need to be produced—and 
will hardly consist in the fact that something is or feels 
awkward for a well-informed agent. Moreover, the objec- 
tion misses its target, divergence, because divergence 
does not entail the alleged source of awkwardness, the 
absence of normative guidance. Divergence allows for 
the possibility that there is a boss: one of the divergers 
or some adjudicator. The possibility is very much alive in 
the literature on normative pluralism, where the view is 
not rare that divergences between kinds of norms coex- 
ist with unequivocal overall norms." 

When it comes to R and M, the observation that diver- 
gence and guidance can coexist gains in stature, since 
one of the two divergers is practical reason itself. There 
is a fairly straightforward sense in which practical reason 
is always in charge. Practical reason deals with practi- 
cal reasons—with all of them. The fascinating question 
whether all of them involve agents’ desires makes no dif- 
ference for the following consideration, which is quite 
general. Neither does it make a difference whether some 
kind of incommensurability threatens to hold among an 
agent’s practical reasons. We may ignore that possibility 
here because we’re grappling with the specter of lacks of 
guidance that originate between R and M. Our question 
is whether the risk of such lacks exists and justifies the 
axiomatic excluding of a divergence of R and M. Surely 
a risk of such lacks could not justify such an excluding 
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if there is lack of guidance even within R. Thus, what 
remains to be looked at is only the other case, in which 
all is well within R: there is some balance of all practical 
reasons that an agent has and thus something that the 
agent has most reason to do. 

And now to the question where the balance leaves 
moral practical reasons. They can relate to the “most rea- 
son” verdict in two different ways but can escape neither 
the verdict nor the guidance it gives. We can understand 
moral practical reasons either as being practical reasons 
of a certain kind (think of yellow bicycles, which are 
bicycles), in which case the balance of all practical rea- 
sons will have taken them into account—or as not being 
practical reasons (think of “root beer,” which is not beer, 
and of “toy money,” which is not money), in which case 
the balance of all practical reasons will not have taken 
them into account. There is guidance by the balance 
either way. It is guidance on the level “most practical 
reason,” a level on which all practical reasons—that is, 
all reasons to do things—have been taken into account. 
That seems guidance enough. 

From the list of objections against divergence, not 
much is left. It seems that we should bemoan, but not 
deny, the existence of divergence. 


8. Conclusion 


We have explored in which sense and why it is in many 
cases rational for people to do what morally they ought 
to do but also why with respect to many cases, even of 
well-informed agents, the diagnosis is controversial. We 
have sided with the common-sense view that perform- 
ing actions that are stupid (to use the laity’s term for 
“irrational”) and performing actions that are morally 
wrong are two very different kinds of shortcomings. The 
action that is stupid can be altruistic, benevolent, benefi- 
cent, and morally right, and the action that is not stu- 
pid can be egoistic, malevolent, maleficent, and morally 
wrong. Given the duality, theoreticians of the rational 
and the moral will keep or turn their spotlights on the 
kinds, mechanics, and extents of convergence and diver- 
gence, including the metanormative challenges posed 
by competing norms. In practice, both desires and ways 
of fulfilling them ought to be shaped—and that “ought” 
is a moral one—so that divergence is reduced. 


Acknowledgments 
More people have given us a hand regarding some aspect 


or other of this chapter than we can list here, and we 
are grateful to every one of them. Special thanks go to 


689 


Kevin Baum, Inga Lassen, Susanne Mantel, Helge Rtick- 
ert, Rudolf SchiiRler, Stephan Schweitzer, and Christian 
Wendelborn. 


Notes 


1. More on doctrines like instrumentalism and on their rivals 
in Hutcheson (1728, section 1 of treatise 2), Millgram (2001), 
Schroeder (2007), and Schmidt (2016). For further references, 
see Fehige (2001, note 1). 


2. For the equating of welfare with desire fulfillment, see von 
Wright (1963, esp. sections 5.9 and 5.11) and Carson (2000, 
chapter 3); see also Wessels (2011), with numerous sources. 
For the claim that pleasure is one kind of desire fulfillment, see 
Fehige (2004, esp. pp. 143-145) and Heathwood (2007), both 
of which give further references. As to normative ethics: utili- 
tarianism and its rivals are sketched in Vaughn (2010/2013, 
chapter 2) and treated more thoroughly in Copp (2006, part 2); 
a helpful introduction to utilitarianism is Bykvist (2010). 


3. Hutcheson (1728) covers in some detail the desires associ- 
ated with the “publick sense,” which is the disposition “to be 
pleased with the Happiness of others, and to be uneasy at their 
Misery” (art. 1.1 of treatise 1), and with the “moral sense,” which 
is the disposition to have moral sentiments. More on the con- 
nections between morality, desire, sentiment, and sympathy 
in Bricke (1996, esp. chapter 6), Fehige (2004), and Fehige and 
Frank (2010). 


4. Single-source approaches to morality have been champi- 
oned, for example, by Arthur Schopenhauer (1841, esp. sec- 
tion 16), who counts on sympathy, and by Francis Hutcheson 
(1725, preface and treatise 2) and David Hume (1751, appendix 
1), who count on the moral sentiments. 


5. The argument is put forth in Fehige (2004); the book 
includes a discussion, in chapter 6, of the limits of even that 
connection between R and M. 


6. Lucid treatments include Hutcheson (1728, esp. art. 1.3 of 
treatise 1 and the beginning of treatise 2) and Sharp (1923, 
section 2). 


7. Gregory Kavka captures the difference with maximum clarity 
when he distinguishes the “Wider Reconciliation Project” (1985, 
section 5) from a narrower one. More on handling egoism and 
its relation to morality can be found in Cholbi (2011). 


8. More on the important distinction, for example, in 
Hutcheson (1728, art. 1.4 of treatise 1), Rashdall (1907, esp. 
pp. 17-18, 28-32), and Schlick (1930, sections 2.6-2.8). 


9. For the first kind of approach, see Fudenberg and Tirole 
(1991, chapter 5, esp. section 5.1.2) and Maschler, Solan, and 
Zamir (2008/2013, chapter 13, esp. sections 13.5 and 13.6). The 
agent-based simulation approach is surveyed by Gotts, Polhill, 
and Law (2003); a telling example is Hegselmann (1998). 
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10. Moral rationalism is also known as the “claim of over- 
ridingness” and is highly controversial; contributions to the 
dispute include Scheffler (1992a, chapter 4), Cholbi (2011, esp. 
section 1), Portmore (2011), and Dorsey (2012). An illuminat- 
ing critical study of ways of engineering convergence is Brink 
(1992). 


11. Peterson (2015) can serve as a gateway to the area; see also 
Dawes (1980) and Binmore (1994). 


12. A good starting point is Gauthier (1986, esp. chapter 6, 
which has the upcoming quotation on p. 158). Appeals to 
“resolute choice” (McClennen, 1985) bear some resemblance 
to Gauthier’s “constrained maximization.” Gauthier later 
espouses a conception of rationality that he acknowledges is 
morally charged to begin with (2013, p. 624); for a similar step, 
see McClennen (2012). 

Other devices, too complex to sketch here, have been 
invoked against the traps: the claim that people who are in 
similar circumstances are bound to act similarly (see Davis, 
1985, and, on “mirror strategies,” J. V. Howard, 1988), the 
proposal to shift our attention from Nash equilibria to “depen- 
dency equilibria” (Spohn, 2009, foreshadowed by Aumann, 
1987), and, although with an emphasis on the explanatory 
rather than the normative dimension, the conceptualization 
of a relevant decision situation as a “metagame” that involves 
partial commitments (N. Howard, 1971, esp. sections 2.5, 3.1, 
3.2) or of the decision-makers as members of a group (Bacha- 
rach, 2006, esp. section 4 of the “Conclusion,” and Butler, 
2012). For critical thoughts on some of the approaches, see, 
e.g., Binmore (1994, chapter 3). 


13. The view that rationality is the corresponding of actions to 
beliefs about reasons can be spelt out in quite different ways; 
see, for example, Scanlon (1998, sections 1.1.3-1.1.5) and Parfit 
(2011, sections 1 and 17). 


14. The claim that there are worldly normative reasons for 
actions has acquired quite a few supporters. Among them 
are Thomas Nagel (1970, esp. chapter 10, and 1986, esp. sec- 
tions 8.4, 8.5, 9.2, 9.3), Thomas Scanlon (1998, sections 1.9— 
1.11), Jonathan Dancy (2000, chapters 2 and 5), Philippa Foot 
(2001, chapter 4), Frederick Stoutland (2001, sections 3.2 and 
3.3), and Derek Parfit (2011, sections 1-15). 


15. The quotations are from Kant (1785/1903, p. 430); for the 
moral impact of autonomy, see, for example, pp. 405, 428-434, 
444, 447-452. Kant later invokes the moral law as a “fact of 
reason” (1788/1908, pp. 31-32, 42-43); whether in doing so he 
renounces, summarizes, or supplements his justificatory efforts 
is controversial. After Kant, attempts to ground morality in the 
respect for rationality itself have been numerous and varied; 
Smith (2011) is one case in point. 


16. The three kinds of prerogatives are considered, one each, 
in Scheffler (1992b, section 1), Mulgan (2001, section 5.5), and 
Nagel (1986, p. 202). For further reflections on possible kinds 
and shapes of prerogatives, see Wessels (2002) and Stroud 
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(2010). Some prerogativists make it clear that anti-divergence 
is where they come from. It is for the sake of keeping M in 
the orbit of R that they conceive of M as partial and as in that 
respect non-utilitarian (see Nagel, 1986, pp. 202-203, and Port- 
more, 2011). 


17. A particularly acute analysis of the connections between R 
and M in the contractarian project is provided by Rainer Trapp 
(1998), who also explains (pp. 339, 356-359) that it will not 
always be rational for a person to do what by contractarian 
lights she morally ought to do. Contractarians who acknowl- 
edge the divergence include Peter Stemmer (2017, pp. 646-648, 
esp. note 34) and Gregory Kavka (1985, pp. 305-308 and, most 
clearly in terms of rationality, section 5). That a contractarian 
like Gauthier might avoid the divergence by operating with a 
modified conception of rationality (Gauthier, 1986, chapter 6) 
is a different matter. 


18. The preceding quotations are from Harman (1975, p. 9); 
the subsequent ones are from Harman (1977, p. 107). Stem- 
mer has retracted the crucial claims (2017, note 34) but used 
to travel a very similar path. For a while, he saw norms geared 
so radically to the addressee’s wanting that even an accidental 
hole in the sanctioning was considered to constitute a hole in 
the norm itself. Insofar as an individual action would not be 
followed by a sanction that the agent herself wants to avoid, 
so the retracted claims run, the norm not to perform that 
action “does not exist,” and the action “is not really forbidden” 
(Stemmer, 2008, p. 181). 


19. That domain-specific “ought” judgments relate to over- 
arching ones, which have the last word, is argued by McLeod 
(2001) and Woods (2018, section 10.2.2). In a similar spirit, 
Case (2016) provides a powerful argument for the conditional 
claim that, if you accept “source pluralism” and “conflict,” 
you are committed to accepting “authoritative adjudication.” 
However, support for the claim that “ought” judgments from 
different domains diverge is wider and comes also from authors 
who deny that there is an adjudicative level—see Baker (2018) 
and the sources given there. 
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Summary 


How are moral reasons to be integrated into empirical 
psychology? The answer depends on our views about 
both morality and reasons for action. As regards morality, 
there is disagreement about whether we should be Ratio- 
nalists or Antirationalists, and hence about whether one 
mark of the moral is that moral requirements entail rea- 
sons for action. As regards reasons for action, although 
it is widely agreed that we have a reason to do what we 
would be motivated to do if we deliberated well, there is 
disagreement about what it is to deliberate well. Should 
we be Humeans about deliberating well or Reasons Prim- 
itivists, or Constitutivists? The main difference between 
these views lies in their accounts of the rational evalu- 
ation of intrinsic desires. Psychologists who talk about 
moral reasons and the philosophers who collaborate 
with them therefore need to be clear about which of 
these assumptions they are making in their research. 


1. Different Views about Morality, Reasons 
for Action, and Their Relationship 


Moral reasons must in some way be integrated into 
empirical psychology, but the manner of their integra- 
tion will depend in large part on which of the compet- 
ing philosophical views about morality and reasons for 
action we should accept. Our views about morality will 
depend on what we think moral knowledge is knowledge 
of, and hence on whether we think that basic moral facts 
are knowable a priori. Our views about reasons for action 
will depend on our understanding of the constitutive 
norms governing belief and desire, and hence on our 
understanding of the functional roles that these states 
play, where their playing these roles is what makes them 
the psychological states that they are. Unfortunately, 
few views about these matters are uncontroversial, so 
how should we proceed? 

At a minimum, psychologists should be explicit 
about which of the controversial views concerning these 


matters their theories and hypotheses assume, and this in 
turn suggests that they should engage with philosophers 
who can help them navigate the conceptual minefields. 
To make clear the nature of these conceptual minefields, 
this chapter will describe two views about morality— 
Rationalism and Antirationalism—and three views about 
reasons for action—Humeanism and two anti-Humean 
views: Reasons Primitivism and Constitutivism—and con- 
sider how these views are to be combined with each other. 
Some criticisms will be offered, but since these criticisms 
are themselves controversial, readers will be left to judge 
the overall merits of the views for themselves. 


2. Morality 


Let’s begin by getting clear about the nature of morality. 
What is the mark of moral principles? 

Moral principles are practical principles that tell us, 
inter alia, what we ought to do. Although there is disagree- 
ment about their substance, it is more or less universally 
acknowledged that the mark of moral principles—that is, 
what distinguishes them from practical principles of other 
kinds like aesthetic principles, principles of etiquette, and 
tules of games—is, inter alia, that they are principles of 
impartial conduct (Gauthier, 1986; Hare, 1981; Scanlon, 
1982). Put slightly differently, the moral viewpoint is the 
viewpoint we adopt when we think of ourselves as just 
one among many, each of whom is to be accorded the 
same consideration as everyone else. It is in this sense 
that moral principles tell us that we ought to treat each 
other, ourselves included, impartially, under some inter- 
pretation of what it is to treat each other impartially. 

Disagreement about the substance of morality, so 
understood, is disagreement about either the scope of 
principles of impartial treatment or the conception of 
impartiality. Perhaps we’re required to treat impartially 
all sentient beings, or all rational agents, or all those to 
whom we can justify our conduct, or all of the members of 
some other group of which we are members. Answering 
this question about the scope of principles of impartial 
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treatment requires us to give an account of which of the 
many kinds of which we are members is the kind in vir- 
tue of which we are owed impartial treatment. The idea 
that moral principles are by nature impartial leaves this 
open. 

Similarly, the idea that moral principles are by nature 
impartial leaves it open whether treating each other 
impartially requires us to give the similar interests of each 
the same weight in decisions about how we act so as to 
promote, as best we can, the welfare of all; to act in ways 
that leave people free to choose how to live on condi- 
tion that they leave others similarly free; or to be able 
to justify our conduct to each of the parties affected by 
our conduct. The idea that moral principles are by nature 
impartial is thus consistent with utilitarianism, deontol- 
ogy, and contractualism and no doubt with many other 
candidate moral views too. 

If the mark of moral principles is, inter alia, that they 
are principles telling us to act impartially, then what 
is ruled out? What is ruled out is the idea that moral 
principles might tell us, at the most fundamental level, 
that certain individuals count more than others simply 
in virtue of being the individuals that they are. It is thus 
a conceptual confusion to suppose that our welfare, our 
freedom, or our recognition by others counts for more 
morally than that of others simply because it is ours. 
Egoism in all its forms is thus a nonmoral view par excel- 
lence. It should therefore come as no surprise that the 
paradigmatic immoral person is someone who is utterly 
selfish. 

In this respect, moral principles stand in stark con- 
trast to principles of epistemic and practical rationality— 
these are the constitutive norms governing belief and 
desire mentioned earlier—at least on the conception of 
such principles we have inherited from Hume (1739- 
1740/1969). This is important because, as we will see 
shortly, it is widely agreed that there is a tight connec- 
tion between reasons for action and how we would be 
motivated to act if our motivations conformed to princi- 
ples of epistemic and practical rationality. The stark con- 
trast between moral principles and Humean conceptions 
of epistemic and practical rationality thus suggests a stark 
contrast between moral principles and Humean concep- 
tions of reasons for action. 


3. Humeanism about Reasons for Action 


According to Humeans, principles of epistemic rational- 
ity specify how we ought to form our beliefs in response 
to the evidence available to us, and practical principles 
specify how we ought to be motivated given our intrinsic 
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desires for different outcomes, which may vary in their 
strength, and our beliefs about which of our options sat- 
isfy these intrinsic desires—let’s call these instrumental 
beliefs—where these may vary in their associated con- 
fidence levels. People are thus epistemically and practi- 
cally rational, according to Humeans, to the extent that 
they have and exercise the capacity to form beliefs and 
be motivated in ways that conform to these principles, 
principles that are themselves thoroughly self-referential. 

Note that Humean principles of practical rational- 
ity provide us with a way of assessing the rationality 
of agents’ motivations in the light of whatever intrinsic 
desires and instrumental beliefs they happen to have 
and that Humean principles of epistemic rationality 
provide us with a way of assessing the rationality of 
agents’ beliefs in the light of the evidence available to 
them. Insofar as attempts have been made to codify 
these Humean principles of epistemic and practical 
rationality, the codification has come in the form of the 
probability calculus as a way of modeling rational belief 
revision and expected utility as a way of modeling ratio- 
nal decision making. 

Importantly, this means that the Humean view that 
there are no principles in terms of which we can assess 
the rationality of an agent’s intrinsic desires has itself 
leaked all the way into theoretical psychology. This is 
not surprising given the very different understandings 
of intrinsic desires, on the one hand, and beliefs, on the 
other, we have inherited from Hume—understandings 
that have become part of common sense. Although 
intrinsic desires and beliefs both have representational 
content, Hume thought that intrinsic desires are unlike 
beliefs in not representing anything to be the case. 
Instead, they represent a way the world could be asso- 
ciated with positive affect, or a way that we would be 
disposed to make the world if we believed we had the 
option to make it that way. Moreover, Hume thought it 
followed from this that intrinsic desires are unlike beliefs 
in another respect as well, namely, in not being sensitive 
to evidence of truth and falsehood, and so in not being 
assessable for their reasonableness. This is why Humeans 
deny that there are any principles in terms of which we 
can assess the rationality of intrinsic desires. 

According to Humeans, intrinsic desires thus stand in 
stark contrast not just to beliefs but also to instrumen- 
tal desires, which they take to be amalgams of intrinsic 
desires and instrumental beliefs, amalgams that are apt to 
cause action. Since the instrumental beliefs that partially 
constitute instrumental desires are sensitive to evidence 
of truth and falsehood about the nature of the possible 
worlds in which we pursue our options, Humeans think 
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that the instrumental desires thus partially constituted 
are also sensitive to evidence of truth and falsehood, and 
hence assessable for their reasonableness. This is a big 
difference between instrumental desires and intrinsic 
desires, but it is a difference explained by the fact that 
Humeans think that the former are, whereas the latter 
are not, amalgams partially constituted by beliefs. 

We have focused on Humean conceptions of epistemic 
and practical rationality because, as signaled earlier, on 
many views of reasons for action, both Humean and 
anti-Humean, there is an intimate connection between 
the reasons for action agents have and the principles 
of epistemic and practical rationality to which they are 
subject. This is because of a platitude about reasons for 
action. Reasons for action are those considerations that 
would motivate us to act if we were to deliberate well, 
where deliberating well is a matter of our meeting three 
conditions: first, our beliefs must conform to all princi- 
ples of epistemic rationality; second, our motivations 
must conform to all principles of practical rationality; 
and third, our beliefs and motivations must conform 
to these principles in circumstances in which the world 
is maximally and reliably revealed to us, that is, in cir- 
cumstances in which we can be certain about everything 
that’s relevant to the formation of our beliefs and resul- 
tant motivations (Korsgaard, 1986; Williams, 1981). 

Note that this platitude about reasons for action helps 
us understand the often-made distinction between subjec- 
tive and objective reasons for action (Sepielli, 2018). Objec- 
tive reasons for action are those considerations that meet 
all three conditions just mentioned, whereas subjective 
reasons for action are those considerations that meet just 
the first two conditions. Subjective reasons for action are 
thus relative to the available evidence, where that evi- 
dence may be misleading. Objective reasons for action, by 
contrast, are relative to the facts. The platitude also helps 
us to understand the commonsense distinction between 
what there is some reason to do and what there is most 
reason to do. Since agents may have multiple motiva- 
tions, some weaker and others stronger, it follows that 
they have some reason to do whatever they would have 
some motivation to do if they deliberated well, where 
that motivation may be weak or strong, and that they 
have most reason to do whatever they would be most 
strongly motivated to do if they deliberated well. 

We are now in a position to see that what gets codified 
in expected utility conceptions of rational decision mak- 
ing is the Humean’s conception of what we have most sub- 
jective reason to do. In what follows, we will mostly ignore 
such reasons and focus on what we have some objective 
reason to do, as this focus will make it easier for us to 
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see the contrast between Humean conceptions of reasons 
for action and moral requirements. Unless flagged other- 
wise, all talk of reasons for action in the remainder should 
therefore be understood accordingly. How do Humeans 
think about such reasons? They think we have (some 
objective) reason to do each of the things that we would 
have some motivation to do if we were to deliberate well 
on the basis of the facts about what would satisfy our vari- 
ous intrinsic desires (Schroeder, 2007; Williams, 1981). 

One immediate consequence of this Humean view 
of reasons for action is that, in certain circumstances, 
there are some people who have no reason at all to act 
in accordance with moral principles. Imagine someone 
utterly selfish whose only intrinsic desire is to promote 
his own welfare. In circumstances in which the evi- 
dence reveals to him that his own welfare won’t be pro- 
moted by his acting impartially, under any conception 
of impartiality, he will violate no principle of epistemic 
or practical rationality by having no motivation to so 
act. According to Humeans, he therefore has no reason 
to act impartially. Whether people have a reason to 
act impartially will therefore depend entirely on what 
intrinsic desires they happen to have and whether their 
acting impartially will lead to the satisfaction of one of 
these desires. 


4. Rationalism versus Antirationalism about Moral 
Reasons for Action 


Is this an objection to the Humeans’ view of reasons for 
action, and hence to their conception of principles of 
practical rationality? Not necessarily. 

I said at the outset that we will consider two views 
of morality. One of these views is Rationalism, the view 
that if there are any moral requirements, then the basic 
requirements are knowable a priori and entail correspond- 
ing reasons for action (Kant, 1785/1948). The other view 
is Antirationalism, which is just the denial of Rational- 
ism (Brink, 1989; Railton, 1986). If Rationalism is true, 
and if those who have no intrinsic desires that would be 
served by their acting impartially are nonetheless subject 
to moral requirements, then this is a decisive objection to 
the Humean view of reasons for action. But if Antiration- 
alism is true, it is no objection at all. 

Moreover, at first sight, there seems to be nothing 
implausible about Antirationalism. Consider norms of 
other kinds. People can be subject to rules of games— 
“Thoroughly shuffle the deck before dealing the cards in 
poker”—and social rules like requirements of etiquette— 
“Reply in the third person to an invitation sent in the 
third person”—without having any reason at all to 
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conform to these rules or requirements (Foot, 1978). 
A magician who has been secretly hired to entertain 
the other players with their card tricks need have no 
reason at all to shuffle the cards when playing poker, 
and a maverick who receives an invitation written in 
the third person need have no reason at all to reply in 
the third person. Since the basic requirements of poker 
and etiquette can change over time, knowledge of such 
requirements also seems to be a posteriori knowledge 
par excellence, not a priori knowledge. Antirationalists 
are therefore within their rights to ask why we should 
suppose that requirements of morality are any different. 

How will Humeans who are Antirationalists think 
about moral reasons? They will think that people have 
moral reasons only contingently, and they will think 
that they have them in virtue of having some intrinsic 
desire that would be served by their acting impartially 
in the sense of “impartiality” picked out by the correct 
conception of morality. Humean Antirationalists may 
disagree among themselves about what the empirical 
facts are that make a conception of morality the correct 
conception. However, they will all agree that in circum- 
stances in which people have no such intrinsic desires, 
they have no moral reasons. 

Note that Humean Antirationalists needn’t deny 
that some people will have moral reasons more robustly 
than others. For example, those who happen to have an 
intrinsic desire to act in accordance with moral require- 
ments will have moral reasons even when their acting 
in accordance with such requirements doesn’t serve 
any other intrinsic desire they happen to have, whereas 
those who lack such a morally loaded intrinsic desire 
will only have reasons to act impartially when they do 
have some other such intrinsic desire. But even those 
who have morally loaded intrinsic desires, and so have 
moral reasons robustly, will still only have such reasons 
contingently, as they wouldn’t have had such reasons if 
they hadn’t had an intrinsic desire to act in accordance 
with moral requirements. 


5. The Problem with Humean Antirationalism 
about Moral Reasons 


How plausible is Humean Antirationalism about moral 
reasons? The main attraction of this view is its modesty. 

As regards psychology, Humean Antirationalism about 
moral reasons assumes only easily understandable con- 
ceptions of belief and desire, ideas that have been codified 
in the probability calculus and expected utility concep- 
tions of rational decision making, and as regards norms, 
it assumes that moral requirements are no different from 


Michael Smith 


other, more familiar norms like rules of games and social 
rules like requirements of etiquette. If we are to reject 
Humean Antirationalism, then moral requirements must 
be very different from these other norms, and belief and 
desire must be rather different from the way they’re ordi- 
narily taken to be. 

Rationalists think that moral requirements are indeed 
very different. The problem, as they see things, is that 
Humean Antirationalism cannot be squared with the 
conditions under which we hold people morally respon- 
sible. Let’s begin with people’s responsibility for having 
knowledge of basic moral facts. We assume that such 
knowledge is within the grasp of anyone with normal 
powers of reasoning independently of where or when 
they grew up. In this respect, it is more like knowl- 
edge of basic mathematical facts. Although this doesn’t 
entail that basic moral facts are, like mathematic facts, 
knowable a priori, it does rule out many of the obvi- 
ous empirical alternatives. For example, if we assume 
that intelligent extraterrestrials would have knowledge 
of basic moral facts, as it seems we do, then that rules 
out moral facts being empirical facts that are peculiar to 
human beings or to life on Earth. 

Now consider people’s responsibility to act in accor- 
dance with moral requirements. When people fail to act 
in accordance with a moral requirement, we take them to 
be fit candidates for blame if they don’t have an exemp- 
tion or an excuse. Exemptions include the fact that they 
lack the normal powers of reasoning required to have 
knowledge of basic moral requirements—perhaps they 
are infants or insane—and excuses include their being 
nonculpably ignorant of the fact that the circumstances 
that they find themselves in are circumstances in which 
the moral requirement applies to them. 

Importantly, however, someone’s lacking any intrin- 
sic desire that would be served by their acting in accor- 
dance with a moral requirement is neither an exemption 
nor an excuse. Imagine an immensely callous person 
who fails to help someone in need when their helping 
them is morally required. The fact that they don’t care is 
neither an excuse nor an exemption for their failure to 
help. Those who fail to act in accordance with a moral 
requirement simply because they lack any desire that 
would be served by their doing so are therefore blame- 
worthy. The question is whether this can be squared 
with the Humean’s conception of moral reasons. 

There is a difficulty here, as it seems that someone 
couldn’t be blameworthy for failing to help if they had 
no reason to help (Darwall, 2006; Portmore, 2011). They 
couldn’t be blameworthy because it would be com- 
pletely unreasonable to expect anyone to do something 
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that they have no reason to do. Someone’s being a fit 
candidate for blame thus seems to presuppose that there 
was at least some reason for them to do what they’re 
blameworthy for failing to do. Indeed, it seems to pre- 
suppose something much stronger, namely, that there 
was a decisive reason for them to do what it would be 
fit to blame them for failing to do. Someone’s failing to 
act in accordance with a moral requirement is in this 
respect very different from their failing to act in accor- 
dance with the rules of a game or with requirements of 
etiquette. 

Failures to abide by rules of games or requirements 
of etiquette do not in general imply that violators are 
fit candidates for blame, as the examples given earlier 
amply illustrate. Of course, blame is appropriate in 
such cases if the violation of the rule of the game or the 
requirement of etiquette is, in the circumstances, also 
the violation of a moral requirement. But that just rein- 
forces the point that there is an important difference 
between moral requirements and other norms like rules 
of games and requirements of etiquette, and that the 
difference lies in the fact that moral requirements do, 
whereas rules of games and requirements of etiquette do 
not, entail reasons for action. 

Here, then, lies the problem with the Humean con- 
ception of moral reasons. According to Humeans, those 
who lack intrinsic desires that would be served by their 
acting in accordance with a moral requirement have 
no reason at all to so act, but this cannot be squared 
with the conditions under which we hold people mor- 
ally responsible. This is because, on the one hand, we 
don’t blame people for acting in ways that they have 
no reason to act, while on the other, we do blame them 
for failing to act in accordance with moral requirements, 
absent an excuse or an exemption, where lacking an 
intrinsic desire that would be served by their so acting 
is neither. Something has to give, and it appears to be 
the Humeans’ view of reasons for action. But what is the 
alternative to the Humeans’ view? 

Let’s return to the platitude about reasons for action. 
It tells us that people have reason to do what they would 
be motivated to do if they deliberated well. Where 
Humeans seem to go wrong is in their conception of 
what it is to deliberate well, particularly in their view 
that there are no principles of practical rationality in 
terms of which we can assess an agent’s intrinsic desires. 
But Humeans are surely right that intrinsic desires don’t 
themselves represent anything to be the case but instead 
represent things as being either a way things could be 
that is associated with positive affect or a way that we 
would be disposed to make them if we believed that we 
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had the option of doing so. If they aren’t truth-assessable, 
then how can there be principles of practical rationality 
in terms of which we can assess their rationality? We 
will consider two answers Anti-Humeans have given 
to this question, a radical answer and a not-so-radical 
answer. Only the radical answer requires us to reject the 
Humeans’ conception of intrinsic desire. 


6. Reasons Primitivism about Reasons for Action 


According to Reasons Primitivists, Humeans go wrong at 
the very beginning (Parfit, 2011; Scanlon, 2013). At the 
most fundamental level, the considerations that support 
the truth of our beliefs do so because they count in favor 
of believing, where counting in favor is a primitive rela- 
tion that considerations stand in to states of believing, a 
relation that is not further analyzable in terms of being 
truth-supporting. If we ask what it is for considerations 
to count in favor of believing, Reasons Primitivists tell us 
that all we can say is that it is for them to provide reasons, 
and if we ask what it is for considerations to provide rea- 
sons, they tell us that it is for them to count in favor. 

Of course, Reasons Primitivists don’t deny that the 
considerations that count in favor of believing support 
the truth of what’s believed. What they deny is just that 
we could analyze the reason-relation in terms of truth- 
supportingness. In their view, it is a substantive truth 
about a certain class of reasons, reasons for belief, that 
they are truth-supporting. We cannot turn this into an 
analysis of the reason-relation, they say, because not 
all of the considerations that count in favor of psycho- 
logical states for which reasons can be given—call these 
judgment-sensitive attitudes—are truth-supporting consid- 
erations. The judgment-sensitive attitudes include not 
just believing but also intrinsically desiring, intending, 
trusting, admiring, fearing, and so on. Although the con- 
siderations that count in favor of believing support the 
truth of what’s believed, the considerations that count 
in favor of intrinsically desiring, admiring, and trusting 
don’t count in favor of the truth of what’s intrinsically 
desired, what’s admired, and what’s trusted. Reasons for 
beliefs are thus outliers in this group. 

Reasons Primitivists thus recommend radical alterna- 
tives to the Humeans’ principles of epistemic and prac- 
tical rationality. In their view, ideally rational people 
are those whose beliefs and intrinsic desires—and their 
intentions, attitudes of trust, feelings of admiration, and 
so forth—come and go to the extent that they take there 
to be considerations that count sufficiently in favor of 
them. Someone who is ideally rational thus acquires 
intrinsic desires, just like they acquire beliefs, when they 
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take there to be sufficient reasons for them, and they 
lose their intrinsic desires, just like they lose their beliefs, 
when they don’t take there to be sufficient reasons for 
them or when they take there to be insufficient rea- 
sons for them. Where Humeans see a striking difference 
between beliefs and intrinsic desires, Reasons Primitiv- 
ists see a striking similarity. 

This, in turn, suggests a very different Reasons Primi- 
tivist account of what it is for there to be (some objec- 
tive) reason for action. Remember again the platitude 
about reasons for action. It tells us that what we have 
a reason to do is what we would be motivated to do if 
we were to deliberate well, where deliberating well is 
a matter of our beliefs conforming to all principles of 
epistemic rationality, our motivations confirming to all 
principles of practical rationality, and our beliefs and 
motivations conforming to these principles in circum- 
stances in which the world is maximally revealed to us. 

Reasons Primitivists think that the first and second 
conditions are met when agents are ideally rational— 
that is, when they have beliefs and intrinsic desires, and 
hence resultant motivations, that they take there to be 
sufficient reasons for—and they think that the third 
condition is met when the considerations that agents 
take to be sufficient reasons for believing and intrinsi- 
cally desiring are sufficient reasons. In other words, 
a crucial aspect of the world that Reasons Primitivists 
think needs to be maximally and robustly revealed to us 
if we are to deliberate well is the normative part, the part 
that concerns which considerations are sufficient rea- 
sons for believing and intrinsically desiring, and hence 
for being motivated. 

The difference between the Humean conception 
of reasons for action and the Reasons Primitivist con- 
ception can thus be summed up as follows. Whereas 
Humeans think that there are reasons for agents to do 
whatever will satisfy their intrinsic desires, Reasons Primi- 
tivists think that there are reasons for agents to do what- 
ever will satisfy the intrinsic desires that they have sufficient 
reason to have. In their view, the concept of a reason is 
therefore polysemous. The primitive reason-relation 
holds between considerations and judgment-sensitive 
attitudes. Since actions are not themselves such attitudes, 
they do not figure as a relatum of the primitive reason- 
relation. The concept of a reason for action is defined in 
terms of the concepts of a reason for believing and a reason 
for intrinsically desiring, where the latter is the primitive 
reason-relation, the relation of counting in favor. This 
relation is primitive because no general account can be 
given of why certain considerations count in favor of 
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judgment-sensitive attitudes rather than others. They 
are just the ones that count in favor, in the sense of pro- 
viding reasons for them. 

What do moral reasons look like, according to Rea- 
sons Primitivists? Agents have moral reasons for action, 
according to Reasons Primitivists, if and only if they 
would be motivated to act impartially, given the cor- 
rect conception of impartiality, if they deliberated well, 
where this requires that there are considerations that 
count sufficiently in favor of their being motivated to so 
act. These moral reasons will be noninstrumental if the 
intrinsic desires that they have sufficient reasons to have, 
and that are satisfied by their acting impartially, them- 
selves have impartial contents; otherwise, they will be 
instrumental. This, in turn, means that agents will have 
moral reasons robustly—which is to say independently 
of there being sufficient reasons for them to have intrin- 
sic desires that just so happen to be served by their act- 
ing impartially—just in case they are noninstrumental. 

Note that this Reasons Primitivist account of what 
it is for agents to have noninstrumental moral reasons 
combines readily with Rationalism. What is it for an 
action of a certain kind to be morally required in cer- 
tain circumstances? The most plausible answer for Ratio- 
nalists to give is that an action of that kind is morally 
required in certain circumstances just in case there is 
a decisive noninstrumental moral reason to perform an 
action of that kind in those circumstances. According 
to the Rationalist who is also a Reasons Primitivist, this 
in turn requires that there be a sufficient reason for an 
agent to have an intrinsic desire with the right kind of 
impartial content, that acting in that way satisfies that 
intrinsic desire, and that that sufficient reason is itself 
strong enough, given the other reasons in play, for the 
agent to be most strongly motivated to act in that way 
in those circumstances. 

The Rationalist Reasons Primitivist thus has no prob- 
lem at all making sense of the conditions under which 
we hold agents responsible. Imagine again a callous per- 
son who is morally required to help someone but who 
fails to help not because he has an excuse or an exemp- 
tion but just because he doesn’t care. We blame him for 
his failure. According to the Rationalist Reasons Primi- 
tivist, we blame him because being morally required to 
help entails that he has a decisive moral reason to help, 
and he has this reason because there is a sufficient rea- 
son for him to have an intrinsic desire with impartial 
content that will be satisfied by his helping, an intrinsic 
desire sufficiently strong to make him most motivated to 
do so. The fact that the callous man lacks any desire that 
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will be satisfied by his helping is thus neither here nor 
there as regards his having such a reason. 

We have spent some time outlining the Reasons Prim- 
itivists’ alternative to the Humeans’ account of what it is 
to deliberate well. The main advantage of Reasons Primi- 
tivism is that it combines so well with Rationalism, as 
this allows it to square with the conditions under which 
we hold people morally responsible. The main disadvan- 
tage is that it does so at the cost of postulating a primitive 
normative relation of a consideration’s counting in favor 
of an attitude. This isn’t just metaphysically profligate 
but also psychologically demanding, as Reasons Primitiv- 
ists have to think that people have attitudes toward this 
primitive relation—the attitude of taking considerations 
to count in favor of their believing and desiring—and 
that they are epistemically and practically rational to the 
extent that their beliefs and desires are sensitive to these 
attitudes. It is thus worth noting that one upshot of Rea- 
sons Primitivism is that those who deny the existence 
of such a primitive relation are apparently irrational by 
default. 

This forces us to ask an obvious question: is a less rad- 
ical alternative available that combines just as well with 
Rationalism? If so, parsimony and plausibility will tell 
in favor of preferring that view to Reasons Primitivism. 


7. Constitutivism about Reasons for Action 


Anti-Humean Constitutivists—hereafter “Constitutivists,” 
for short—think that such an alternative is available (Kors- 
gaard, 1986; Smith, 2013). In their view, the Humeans’ 
principles of epistemic and practical rationality are fine 
as far as they go. The problem isn’t that we should reject 
these principles, as Reasons Primitivists do, but rather that 
we need to add more principles. But what are these extra 
principles, and why should we add them? 

Constitutivists begin by reminding us that the com- 
plete set of rational principles governing beliefs and 
desires are those principles, whatever they are, that 
fix the roles these states play when we deliberate well. 
They then point out that it follows from this that if hav- 
ing certain intrinsic desires is itself necessary in order 
to deliberate well, then these intrinsic desires will be 
rationally required. In their view, Humeans therefore go 
wrong in moving from the premise that intrinsic desires 
cannot be assessed for their truth to the conclusion that 
they cannot be assessed for their reasonableness. They 
point out that they would still be so assessable if satis- 
fying them were partially constitutive of what it is to 
deliberate well. 
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For Rationalism to be true, intrinsic desires with 
impartial contents would be required for us to delib- 
erate well. Is this plausible? Constitutivists think it is. 
They point out that deliberating well is a robust activ- 
ity, not something that we manage to do by chance or 
happenstance. This is why one of the conditions on 
correct deliberation is that the world reliably manifests 
itself to us. But this means that other wordly conditions 
are necessary too. After all, we will not deliberate well if 
we are in the company of others who deceive us, or if 
we deceive ourselves, or if others coerce us, or we coerce 
ourselves, or if others stand idly by while we succumb 
to some preventable deterioration in our deliberative 
capacities, or if we stand idly by ourselves. Nor will we 
deliberate well if we or others aren’t reliably disposed not 
to deceive, coerce, or stand idly by while we succumb 
to some preventable deterioration in our deliberative 
capacities. 

What this suggests to Constitutivists is that deliber- 
ating well must take place in a social context in which 
others deliberate well too, a social context in which each 
deliberator has a strong intrinsic desire that they do not 
interfere with anyone’s exercise of their deliberative 
capacities—neither their own nor anyone else’s—and 
a strong intrinsic desire that they do what they can to 
ensure that everyone has deliberative capacities to exer- 
cise. Let’s call these “the desires to help and not inter- 
fere.” The role of these intrinsic desires to help and not 
interfere, to repeat, is to make sure that when we act, we 
not only don’t undermine but also sustain the posses- 
sion and exercise of the deliberative capacities that make 
it possible for us to deliberate well. 

Note that the intrinsic desires to help and not inter- 
fere have impartial contents. If having such intrinsic 
desires is constitutive of deliberating well, then it fol- 
lows that all agents, independently of having reasons to 
satisfy whatever other intrinsic desires they might hap- 
pen have, will have noninstrumental moral reasons to 
help and not interfere. Since all agents have such moral 
reasons, Constitutivism entails Rationalism, with moral 
requirements understood in terms of reasons in the way 
suggested earlier. Constitutivism therefore squares well 
with the conditions of moral responsibility. 

Constitutivism also entails that, to the extent that 
people’s satisfying whatever intrinsic desires they hap- 
pen to have doesn’t undermine anyone’s helping and 
not interfering, they will also have reasons to satisfy 
whatever intrinsic desires they happen to have. This is 
why Constitutivism is a less radical alternative to the 
Humeans’ account of what it is to deliberate well. It tells 
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us that it is rationally permissible to satisfy whatever 
intrinsic desires we happen to have, so long as doing 
so is consistent with our acting on our moral reasons to 
help and not interfere when these reasons are decisive. 

We saw earlier that the mark of moral principles is that 
they are impartial, where this leaves it open what impar- 
tiality consists in. Note that Constitutivism takes a stand 
on this issue. The idea that we all have reasons to satisfy 
whatever intrinsic desires we happen to have, so long as 
our doing so is consistent with our acting on our moral 
reasons to help and not interfere when these reasons are 
decisive, sits happily alongside the familiar deontological 
conception of moral principles as protecting the freedom 
of each person to live a life of their own choosing, so long 
as their doing so is consistent with everyone else’s doing 
the same thing. According to this view, welfare is not of 
intrinsic moral significance. Welfare matters morally only 
to the extent that it affects freedom. 


8. Conclusion 


The question we began with is how moral reasons are to 
be integrated into empirical psychology. We have seen 
that the answer we give to this question will depend on 
our views about both morality and reasons for action. 

As regards morality, although it is widely agreed 
that the mark of moral requirements is their impartial- 
ity, there is disagreement about whether we should be 
Rationalists or Antirationalists and hence whether a fur- 
ther mark of moral requirements is that they entail rea- 
sons for action, as Rationalists think they do. As regards 
reasons for action, although it is widely agreed that we 
have a reason to do what we would be motivated to do 
if we deliberated well, there is a great deal of disagree- 
ment about what it is to deliberate well. Should we be 
Humeans about deliberating well, Reasons Primitivists, 
or Constitutivists? The main difference between these 
views lies in their very different accounts of the rational 
evaluation of intrinsic desires. 

Given the disagreement, psychologists who talk about 
moral reasons need to make clear the assumptions they 
make about morality and reasons for action. Do they 
assume Rationalism or Antirationalism about morality, 
and do they assume Humeanism, Reasons Primitivism, 
or Constitutivism about reasons for action? If they make 
these assumptions clear, then, with the passage of time, 
the answer to a further important question will hope- 
fully emerge: do certain of these assumptions rather than 
others make for a more productive research program? The 
answer to this question will be of great significance to 
psychologists and philosophers alike. 


Michael Smith 
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12.3. The Psychology and Rationality of Moral Judgment 


Alex Wiegmann and Hanno Sauer 


Summary 


The field of moral psychology has become increas- 
ingly popular in recent years. This chapter focuses on 
two interrelated questions: first, how do people make 
moral judgments? It will review the most prominent 
theories in moral psychology that aim to characterize, 
explain, and predict people’s moral judgments. Second, 
how should people’s moral judgments be evaluated in 
terms of their rationality? This question is approached 
by reviewing the debate on the rationality of moral judg- 
ments and moral intuitions, which has been strongly 
influenced by findings in moral psychology but also by 
recent advances in learning theory. 


1. Rationality and Morality 


The rationality of moral judgment is one of the most 
contested issues in moral psychology. In this chapter, we 
will outline the cutting edge of contemporary work in 
moral psychology, convey a sense of the most promising 
recent developments, and explain where this research 
is currently heading. We will also address some of the 
philosophical implications these theories are supposed 
to yield: Are our moral intuitions robust and trustwor- 
thy? Or are they frail and unreliable? Ultimately, these 
normative questions regarding the rationality of moral 
judgment are what most philosophers and probably also 
many psychologists are interested in. 

In the first part, we will summarize theories that 
make specific predictions about which moral judgments 
people are likely to make in response to which types 
of cases and what the psychological processes are that 
explain the patterns so discovered. In particular, we will 
explain how a theory based on the distinction between 
model-free and model-based information processing and 
decision making has come out as the main contender 
among such predictive theories. In doing so, we will 
integrate both insights from computational approaches 


to reinforcement learning as well as recent developments 
in dual-process models of cognition. This concludes the 
first, descriptive part of our overview. 

The second, normative part addresses the issue of the 
rationality of moral judgments head-on.' The recent 
debate suggests that this question might not be best 
addressed in terms of the psychological foundations 
of moral cognition: Are moral judgments intuitive or 
controlled? Are they based on emotion or reason? Does 
reasoning produce, or merely rationalize, moral judg- 
ments? Rather, addressing the question of whether the 
processes generating moral beliefs are capable of incor- 
porating and updating on morally salient information, 
that is, capable of learning, seems the most promising 
line of research. We will explain why some authors argue 
that the sophistication of processes of moral learning 
provides reasons for optimism regarding the rationality 
of moral judgments and why some authors continue to 
strike a more pessimistic note. 


2. Predictive Theories of Moral Judgment 


In this descriptive part of the chapter, we will review 
three prominent predictive theories of moral judg- 
ments, namely, Greene’s dual-process model, Mikhail’s 
universal moral grammar theory, and Cushman’s and 
Crockett’s model-free versus model-based approach, 
with a focus on the latter since it is the most recent and 
probably the most promising theory. It should be noted 
that the distinction between “descriptive” and “norma- 
tive” accounts of moral cognition is at least somewhat 
artificial, as some of the models discussed in the former 
also have bearing on the latter issue. What we mean to 
emphasize here is that some theories in moral psychol- 
ogy make specific, testable predictions regarding how 
subjects will actually respond to certain issues and why, 
while the latter focus on deriving possible normative 
conclusions from an overall assessment of the state of 
the art in the psychology of moral judgment. 
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2.1. Dual-Process Models 

The contemporary debate on the rationality of moral 
judgment is dominated by dual-process models of moral 
cognition (Greene, 2013). According to such models, 
cognitive operations are carried out by two fundamen- 
tally different types of processing, one quick, intuitive, 
automatic, evolutionarily old, and frequently uncon- 
scious (System 1), the other slow, effortful, analytic, evo- 
lutionarily young, and consciously controlled (System 2; 
Evans & Stanovich, 2013; Kahneman, 2011). 

Some of these theories hold that moral judgment 
is System 1 all the way down (see section 3.1). Others 
have argued that the two types of processing can be 
mapped onto two different subsets of moral judgment: 
deontological and consequentialist moral judgments 
(Greene, 2008, 2013, 2014). This distinction is imported 
from philosophical moral theory and refers, roughly, 
to the views that the outcome of an action alone deter- 
mines its moral status (consequentialism) or that other 
factors—such as a person’s intentions or the way an out- 
come was brought about (e.g., as a means or a foreseen 
side effect)—matter as well (May, 2014b). This alterna- 
tive is typically operationalized by classifying subjects’ 
responses to certain well-known moral problems (“trol- 
ley cases”; for an overview, see Waldmann, Nagel, & 
Wiegmann, 2012), in which participants have to imag- 
ine a train about to run over five people and are asked 
whether it would be OK to perform a certain action in 
order to save the lives of the five people. The two most 
prominent variants of such trolley cases are usually 
referred to as Switch and Push, respectively. In Switch, 
one can, by flipping a switch, divert the train onto a 
different track, where it would run over and kill a single 
person. In Push, the train can be stopped by pushing 
a large man, who is standing on a bridge crossing the 
tracks, in front of it. While interventions in these cases 
(flipping the switch and pushing the man, respectively) 
are classified as consequentialist responses, not perform- 
ing the action necessary to save the five people is usually 
considered the deontological option.” 

Some want dual-process accounts to debunk deontologi- 
cal theories and vindicate consequentialist ones. The former 
are purportedly based on crude, alarm-like emotions (see 
also the moral-heuristics approach by Gigerenzer, 2008, 
or Sunstein, 2005) and the latter on flexible, sophisticated 
cost-benefit analyses (Greene, 2008). This pattern seemed 
to be reflected by patterns of brain activation, differences 
under cognitive load, and greater response times for con- 
sequentialist judgments, indicating controlled intuition 
override (Greene, Morelli, Lowenberg, Nystrom, & Cohen, 
2008; Greene, Nystrom, Engell, Darley, & Cohen, 2004; 
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Greene, Sommerville, Nystrom, Darley, & Cohen, 2001). 
Moreover, the etiology of deontological intuitions seemed 
to show that they pick up on morally irrelevant differences 
such as how “up close and personal” a person is harmed 
(Greene et al., 2001). 

Criticism of the dual-process model has focused on 
four issues: first, that the empirical data from neuro- 
imaging (Machery, 2014; Poldrack, 2006; Sauer, 2012b) 
or response times (McGuire, Langdon, Coltheart, & 
Mackenzie, 2009) are more ambiguous than originally 
thought. Second, some have argued that the empirical 
data from neuroscience or certain evolutionary con- 
siderations play no actual role in the argument against 
deontology (Berker, 2009; cf. Greene’s [2010] response). 
Third, it’s not obvious why the emotional or intuitive 
basis of deontological judgments should pose any epis- 
temic problem for them at all (Railton, 2014). Fourth, 
and most important, many have argued that even if the 
empirical data are sound, and even if they do the moral 
heavy lifting, and even if they do identify a rationally 
defective process, it remains unclear whether they really 
target deontological theories. The reason for this doubt 
is that the link between consequentialist moral judg- 
ment and controlled cognition can be reversed when 
the consequentialist option is the intuitive one and the 
deontological option requires override (Kahane et al., 
2011). Moreover, in order for people’s moral judgments 
to count as genuinely consequentialist, they would have 
to stem from an impartial concern for the greater good. 
This seems not to be the case (Kahane, Everett, Earp, 
Farias, & Savulescu, 2015). In general, the Trolleyologi- 
cal paradigm in moral psychology tends to misconstrue 
consequentialism (Kahane et al., 2018; but see also Con- 
way, Goldstein-Greenwood, Polacek, & Greene, 2018; 
see also endnote 2). 


2.2 Universal Moral Grammar 
The dual-process model assumes that moral judgments 
result from domain-general processes that are not spe- 
cific to morality. A further possibility is that people are 
endowed with an innate moral faculty that allows them 
to cognitively structure actions in terms of moral cat- 
egories and to assign deontic statuses (right or wrong, 
permissible or impermissible) to those actions depend- 
ing on this structure. Drawing on ideas pioneered in a 
linguistic context, various authors have tried to show 
that moral cognition is based on such a universal moral 
grammar (Hauser, Young, & Cushman, 2008; Mikhail, 
2007, 2011; cf. Prinz, 2008). 

People start to appreciate certain moral distinctions 
at a very early age, many of those distinctions seem 
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to be culture-invariant, and people have a hard time 
articulating the rules structuring their moral thinking 
(Hauser, Cushman, Young, Jin, & Mikhail, 2007). Also, 
some patterns in people’s moral judgment are difficult 
to account for from within a dual-process framework, 
such as the distinction between harming as a means 
and harming as a foreseen side effect (e.g., Switch ver- 
sus TrapDoor; see next section). Primarily, however, the 
nativist case for a universal grammar is supported by 
so-called poverty-of-the-stimulus arguments, according 
to which the moral rules that implicitly inform people’s 
moral judgments are too subtle to be read off the scarce 
available evidence; subjects couldn’t learn these rules 
empirically. We will return to this question after a short 
detour. 


2.3. Model-Free versus Model-Based Moral Judgment 
Crockett (2013) and Cushman (2013) independently 
developed a dual-system account of moral psychology 
that has already had an impact on the field of moral psy- 
chology and will likely continue to do so (e.g., Greene, 
2017; Railton, 2017). According to this approach, two 
distinct systems evaluate actions: while the model-based 
system selects actions based on inferences about its 
(expected) outcomes in a specific situation, the model- 
free system assigns value to actions intrinsically, based 
on past experience (Crockett, 2013; Cushman, 2013). 

Such dual-system frameworks have been argued to be 
of great value for studying and understanding judgment 
and decision making in general (Evans & Stanovich, 
2013; Kahnemann, 2011; see Kruglanski & Gigerenzer, 
2011, for a critical assessment), and Greene’s emotional- 
versus-cognitive framework has been very influential in 
the field of moral psychology (e.g., Greene et al., 2001). 
However, it has been argued that this particular dis- 
tinction of processes (i.e., emotional versus cognitive) 
might not be fully adequate (Cushman, 2013; Pessoa, 
2008) and computationally too crude (Crockett, 2013; 
Mikhail, 2007; see Crockett, 2016, for the advantages of 
formal models) to characterize and explain moral judg- 
ments. Crockett and Cushman’s distinction of action 
evaluation versus outcome evaluation is supposed to 
avoid the shortcomings of Greene’s characterization of 
the two systems while keeping the benefits of a dual- 
system framework (Greene [2017] seems to agree). Find- 
ings from studies dissociating the aversiveness of actions 
from outcomes provide some initial evidence for separate 
action- and outcome-based (model-free versus model- 
based) value representations (Cushman, Gray, Gaffey, & 
Mendes, 2012; Miller & Cushman, 2013; Miller, Hanni- 
kainen, & Cushman, 2014). 
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The distinction between action- and outcome-based 
value representations resembles a distinction from com- 
putational approaches to reinforcement learning (cf. 
Cushman, 2013), and recent advances in neuroscience 
have identified corresponding neural signatures (Dayan & 
Niv, 2008; Dolan & Dayan, 2013; Glascher, Daw, Dayan, 
& O'Doherty, 2010; Wunderlich, Dayan, & Dolan, 2012). 
Reinforcement learning investigates how agents (humans, 
rats, algorithms) can learn to predict the consequences of 
and optimize their actions in interaction with the envi- 
ronment (Dayan & Niv, 2008). Learning algorithms can 
be classified broadly as model free or model based (Sut- 
ton, 1988; Sutton & Barto, 1999, 2018), depending on 
how they learn and select actions in a given situation (for 
an example in human behavior, see Dayan & Niv, 2008). 
Put simply, a model-based algorithm builds a causal repre- 
sentation of its environment, uses this model to consider 
the whole range of effects that different actions would 
have, and chooses the action with the highest expected 
value. In contrast, a model-free algorithm does not rely 
on a model of the environment and thus cannot make far- 
sighted choices. In a given situation, it simply selects the 
action with the best track record, which can be acquired 
by rather simple means such as prediction error (Glascher 
et al., 2010) and temporal difference learning (Sutton, 
1988; cf. Cushman, 2013). Although a model-free algo- 
rithm is computationally cheap, it can often lead to suc- 
cessful behavior. However, if the environment changes, it 
cannot adapt to these changes as readily as the compu- 
tationally rather expensive model-based algorithms since 
it needs to gather experience in the new environment 
to update the track record accordingly (Crockett, 2013; 
Cushman, 2013; Greene, 2017). 

Can the model-free-versus-model-based (or action- 
versus-outcome) framework explain prominent find- 
ings in moral psychology? Let us start with applying the 
framework to three variants of the well-known trolley 
dilemma: Switch, Push, and TrapDoor (the latter being 
similar to Push, but rather than the heavy person being 
pushed from the bridge, the flip of a switch opens a trap 
door through which the large man falls onto the tracks). 
The relation among the approval rates for the respective 
actions in these dilemmas is Switch > TrapDoor > Push 
(Mikhail, 2007, 2011). The model-free-versus-model- 
based framework can provide a straightforward explana- 
tion for the fact that Push is usually evaluated worse than 
the other two cases: although the model-based evalua- 
tion of the actions does not differ (since they lead to the 
same outcome), pushing a person usually leads to nega- 
tive experiences (punishment, distress cues), whereas 
flipping a switch is not associated with a negative action 
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value. In other words, the model-free evaluation prefers 
flipping a switch to pushing a person (Crockett, 2013; 
Cushman, 2013). Things get more complicated when 
comparing the Switch and the TrapDoor variant. Here, 
not only the outcome (saving five lives) but also the 
required action (flipping a switch) is the same. What 
differentiates these two cases is the causal role of the 
victim. In Switch, the one person getting hit by the 
train is not causally necessary for saving the five lives 
(imagine the one person jumping off the track before 
getting hit), whereas in TrapDoor, the five people could 
not be saved without making use of the heavy per- 
son as a trolley stopper (Mikhail, 2007; Wiegmann & 
Waldmann, 2014). The model-free-versus-model-based 
framework attempts to capture this difference by postu- 
lating that the model-based system represents harming 
a person as a subgoal in TrapDoor but not in Switch. 
Subgoals, in turn, can be treated by the model-free sys- 
tem as if they were actions—and if the subgoal is aver- 
sive, as it is in the case of a person getting hit by a train, 
the model-free system assigns a negative value to it.’ 
It is this negative value assignment of the model-free 
system that differentiates side effect cases from means 
cases and can explain why the latter are often judged as 
more aversive. A further popular finding in moral psy- 
chology is that usually people consider harmful actions 
worse than omissions with identical effects (Baron 
& Ritov, 2004). This asymmetry can be explained by 
pointing out that model-free values are only assigned 
to actions but not to the always available option of not 
performing an action. 

To sum up, the model-free-versus-model-based account 
by Crockett (2013) and Cushman (2013) offers a promis- 
ing domain-general theory of moral judgment. 


3. The Rationality of Moral Judgment 


In the first part of this chapter, we described prominent 
theories, findings, and developments in moral psychol- 
ogy. Not surprisingly, such insights came along with 
discussions about how to evaluate people’s moral judg- 
ments and moral intuitions. In this second part of the 
chapter, we will address this normative question by first 
retracing the debate on the rationality of moral judg- 
ments from the beginning of moral psychology to the 
current state of the field on a rather abstract level. We 
then narrow down the considered time frame and con- 
tent by focusing on the rationality of moral intuitions 
and how this debate has evolved in recent years due to 
advances in learning theory. 
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3.1. Classic Rationalism, Social Intuitionism, 
and Sentimentalism 


Classical rationalism According to what has been the 
dominant paradigm in moral psychology for the sec- 
ond half of the 20th century, the rationality of moral 
judgment—or lack thereof—has to be sought in how 
the capacity for moral judgment and reasoning devel- 
ops. Moral reasoning becomes increasingly sophisticated 
with ontogenetic stages. While infants may start out 
with a preconventional moral code in terms of external 
norms and sanctions, older children construe morality 
as a conventional matter of playing one’s role and stick- 
ing to the rules; with adolescence, some people come 
to see morality as a postconventional affair regulated by 
universal principles (Kohlberg, 1969). 

On the other hand, one of the hallmarks of moral 
judgment—such as the distinction between moral and 
conventional norms—is already appreciated at a very early 
age (Smetana & Braeges, 1990; Turiel, 1983), and the Kohl- 
bergian account implicitly operates within a male perspec- 
tive, thereby dismissing moral outlooks that are focused on 
maintaining a complex web of social interactions between 
concrete individuals, rather than on preserving abstract 
rights, principles of justice, or procedures of deliberation, 
as somehow undeveloped and immature (Gilligan, 1982). 


The antirationalist turn With the beginning of the 
21st century, rationalist dominance began to erode. The 
social intuitionist model, for instance, holds that moral 
judgment is a largely intuitive and automatic enterprise; 
reasons are typically produced post hoc, if at all; and when 
they are, it is not in the unmotivated pursuit of truth but 
for the purpose of social persuasion (Haidt, 2001; Haidt 
& Bjorklund, 2008). When people’s moral judgments are 
debunked, they defend rather than question them. When 
this turns out to be unsuccessful, they don’t abandon 
their beliefs but enter a state of being morally dumbfounded. 
Moral judgment, it now seemed, is not just largely intui- 
tive, but which intuitions people have is due to differences 
in their moral foundations, with important political conse- 
quences (Graham, Haidt, & Nosek, 2009; Iyer, Koleva, Gra- 
ham, Ditto, & Haidt, 2012; cf. Curry, 2016; Sauer, 2015). 


Sentimentalism Others are less concerned with the 
proximal processes that yield particular moral judgments 
than with the psychological foundations of valuing itself. 
According to sentimentalism about moral judgment, val- 
ues are distally grounded in (dispositions for) emotional 
reactions (Nichols, 2004; Nichols, Kumar, Lopez, Ayars, 
& Chan, 2016; Prinz, 2006, 2007, 2016). The empirical 
evidence suggests that moral judgments are constituted 
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by emotions, because emotions are both necessary and 
sufficient for moral judgment. Conditions such as psy- 
chopathy suggest that without the appropriate affective 
reactions, moral judgment is impaired (Blair, 1995). A 
battery of studies suggests that changing someone’s emo- 
tions affects their moral beliefs. Dispassionate moraliz- 
ing seems to be nonexistent. On the other hand, both 
the interpretation of this evidence (Sauer, 2012a) and its 
robustness have been called into question. The effect of 
(incidental) affect on moral judgment seems to be mea- 
gre (Landy & Goodwin, 2015; May, 2014a). Psychopaths 
do master the moral-conventional distinction after all, 
despite their affective impairments. 


Rationalist replies Partly due to these developments, 
rationalists about moral judgment have started to speak 
up again. For some, the issue is partly conceptual: moral 
judgments must in some way be responsive to reasons; 
otherwise, they don’t qualify as genuinely moral (Ken- 
nett & Fine, 2009). Sentimentalists are also accused of 
misidentifying their subject matter. What they have 
studied are time-slice responses to far-fetched toy prob- 
lems (Gerrans & Kennett, 2010) rather than people’s 
temporally extended moral agency. Others emphasize 
that rational processes can “migrate” into intuitive ones 
(Sauer, 2017). Still others have offered hybrid theories 
according to which moral judgments are compound 
cognitive-motivational states in which belief and emo- 
tion homeostatically cluster together (Kumar, 2016). 

A third interesting development is due to research 
on how moral judgment affects various other, seemingly 
nonmoral, cognitive domains. Many studies conducted 
in the so-called experimental philosophy (Kauppinen, 
2007) paradigm, for instance, focus on how causal or 
social cognition lies downstream from moral cognition: 
the moral, evaluative, or otherwise normative beliefs 
people hold influence their judgments about what causes 
what (Knobe & Fraser, 2008), who did what intention- 
ally or not (Knobe, 2003), or what someone’s personal 
identity is (Newman, De Freitas, & Knobe, 2015). This 
debate, too, has led to interesting discussions regard- 
ing the legitimacy of such an influence (Sauer & Bates, 
2013) and whether it makes sense for us to be “moral- 
izers through and through” (Knobe, 2010). 


3.2 Moral Intuitions and Rational Learning 

Recently, the debate on the rationality of moral judg- 
ments has focused on moral intuitions. Do our moral 
intuitions track morally relevant features in a reasonable 
way, suggesting that we should grant them high evi- 
dentiary status and be optimistic about their ability to 
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successfully navigate us through, and solve problems in, 
the social world? Or does the relevant evidence advise us 
to paint a darker picture? In this section, we trace how 
these discussions have evolved in recent times due to 
advances in understanding the origins of our intuitions 
in general and of our moral intuitions in particular. For 
what follows, we closely follow Haidt’s (2001, p. 818) 
characterization of the subject matter, according to 
which moral intuition is the sudden appearance in con- 
sciousness of a moral assessment, including an affective 
valence (good-bad, like—dislike), without any conscious 
awareness of having gone through steps of searching, 
weighing evidence, or inferring a conclusion—and these 
moral intuitions are assumed to play a crucial role for 
moral judgments and decisions, although they can be 
overridden by other considerations. 

Especially with the publication of the seminal articles 
by Haidt (2001) and Greene and colleagues (2001) at the 
beginning of the 21st century, the hitherto dominant 
rationalist view of moral judgments (Kohlberg, 1969; 
Piaget, 1932; Turiel, 1983) was seriously called into ques- 
tion by highlighting the role of moral intuitions, which 
were characterized as a rather bad guide for moral judg- 
ments and decision making. Moreover, this view seemed 
to be well supported by decades of psychological research 
indicating that intuitions in general are often misleading 
(Kahneman, 2003, 2011). 

Relatedly but separately, there are interesting recent 
developments in philosophical moral epistemology that 
can both learn from and inform current psychological 
research. In particular, philosophers are keen to explore 
the issue of whether we can trust our moral intuitions and 
under what conditions and whether doing so requires a 
robust notion of mind-independent moral truth. Many 
philosophers argue that, despite what we know about 
the distal and proximal origins of our moral beliefs, we 
need not abandon most, or at least only some, of them 
(Bengson, Cuneo, & Shafer-Landau, 2019; Enoch, 2011; 
Huemer, 2008). 

Recent empirical accounts of intuitive cognition also 
seem to somewhat vindicate this philosophical opti- 
mism regarding the rationality of moral judgment. Due 
to advances in research on the computational under- 
pinnings of learning, especially in reinforcement learn- 
ing (e.g., Mnih et al., 2015) and Bayesian learning (e.g., 
Tenenbaum, Kemp, Griffiths, & Goodman, 2011), it 
does not seem unreasonable to assume that our intu- 
itions are the output of highly sophisticated learning 
systems that are rather flexible and domain general (cf. 
Railton, 2017). The findings suggest that our intuitions 
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can be finely attuned to even subtle contours of the deci- 
sion landscape and often align with normative models 
of judgment and decision making such as, for example, 
Bayesian updating, although we often do not have delib- 
erative access to these learning processes. 

These advances in understanding the origins of our 
intuitions have been used by several authors to argue 
that the rather pessimistic assessment of moral intuitions 
may have to be revised as well. Perhaps most promi- 
nently, Railton (2014, 2017) argues that some popu- 
lar findings in moral psychology that have been used 
to paint people’s moral intuition in an irrational light 
should be reinterpreted. For instance, it was argued that 
people enter a state of moral dumbfounding (see above) 
in response to scenarios featuring taboo cases (such as 
sibling incest) even when most plausible reasons against 
the taboo action have been ruled out by the details of 
the vignette (such as consensual sex between siblings 
with birth control and in secret). However, rather than 
characterizing our aversive responses to such taboo cases 
as misguided, they could be considered well attuned to 
the fact that the two siblings played Russian roulette 
with their emotional health—the fact that everything 
turned out fine does not justify judging their risky action 
as “okay” (cf. Jacobson, 2012; for criticisms of the moral 
dumbfounding effect, see Guglielmo, 2018; Royzman, 
Kim, & Leeman, 2015). In a similar vein, Allman and 
Woodward (2008; see also Woodward & Allman, 2007) 
argue that our affective intuitions often provide us with 
important information in moral settings, such as signal- 
ing the intentions of others and their likely responses to 
our actions, and they can thereby lead to better moral 
decision making. Their view is underscored by findings 
demonstrating that people with damages in areas cru- 
cial to emotional processing exhibit moral behavior and 
decision making that would not be considered appropri- 
ate by any reasonable standard (e.g., Damasio, 1994). 

The view that moral reasoning can “migrate” into 
people’s moral intuitions is now gaining momentum 
(Sauer, 2017). To give a concrete example of how a ratio- 
nal learning process can lead to moral intuitions, let us 
consider a study by Nichols and colleagues (2016), who 
have argued that the acquisition of deontological rules 
may be based on a Bayesian principle called the “size 
principle.” To illustrate this principle, imagine that there 
are four fair dice, each with a different denomination: 
4, 6, 8, and 10. Out of your sight, one dice is picked at 
random and rolled 10 times, and you are told the out- 
comes: 2, 2, 1, 4, 3, 1, 2, 2, 3, and 4. Your task is to fig- 
ure out which dice has been rolled. The formula of the 
size principle provides exact probabilities for the four 
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hypotheses (4, 6, 8, 10) and favors the narrowest one 
(4), which corresponds to our intuitive assessment that 
we would expect some outcomes to be greater than 4 if 
a dice with a denomination greater than 4 was rolled. 
The size principle is able to explain children’s fast learn- 
ing of word categories: a few positive examples of dif- 
ferent dogs that are called “dogs” are sufficient to infer 
that the term refers to dogs and not to a broader cat- 
egory of animals, and some examples of Dalmatians are 
sufficient to let them infer that the term “Dalmatian” 
refers to this breed of dogs and not to dogs in general 
(again favoring the narrower hypothesis). Nichols and 
colleagues provide experimental evidence indicating 
that deontological principles, such as the act-allow and 
the intended-foreseen distinction, can be learned by the 
same mechanism. Moreover, given the learning input 
children receive (as indicated by a corpus analysis), they 
would naturally acquire such deontological rules by 
approximating Bayesian inference—it might even be sta- 
tistically irrational for them to instead infer utilitarian 
rules. Hence, Nichols and colleagues provide a rational 
learning mechanism for the acquisition of deontologi- 
cal principles that can undercut attempts to characterize 
deontological rules and principles as the product of irra- 
tionality. Moreover, it also offers an empiricist explana- 
tion of how rational learners can retrieve sophisticated 
moral rules from seemingly limited information in a 
way that escapes nativist arguments from the poverty of 
moral stimuli. 

Let us assume, for the sake of the argument, that intu- 
itions are the output of rational learning mechanisms. 
Does this fact mean that we should stop worrying that 
moral intuitions might be misleading and grant them 
moral authority? Some authors believe that such an 
optimistic view would not be justified. For instance, 
Gjesdal (2018) points out that since the learning mecha- 
nisms underlying our moral intuitions are domain gen- 
eral, they are potentially sensitive to morally relevant 
features—but they might also pick up morally irrelevant 
or even immoral information.* We should not consider 
them as morally authoritative on their own, Gjesdal 
argues, as long as we cannot rule out the possibility 
that moral intuitions are affected by morally nonrele- 
vant features. This skeptical stance gains traction by the 
plausible assumption that our learning systems are well 
attuned and sensitive to information that is important 
for an agent’s welfare. The reason is the—unfortunate— 
fact that what promotes an agent’s welfare and what 
morality requires from her often come into conflict.® 
Hence, it cannot be ruled out that our moral intuitions 
are tainted by self-interested considerations. One might 
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address this worry by pointing out that in an appropri- 
ately structured environment, our learning mechanisms 
and moral intuitions could become properly attuned to 
morally relevant features. For instance, we might learn 
that a certain degree of impartiality is necessary for us to 
successfully navigate through the social world, and prej- 
udices might be reduced by coming into contact with 
the respective people. But making the case for moral 
intuitions in this way would let it rest on shaky ground, 
since it is far from obvious that our environment was, is, 
or will become structured in a way that would guarantee 
that our moral intuitions are (or will become) adapted 
in a morally appropriate way. Hence, Gjesdal concludes, 
the finding that our intuitions are rationally attuned to 
reality does not warrant that they are morally appropri- 
ate. Greene (2017) seems to agree with this assessment 
and also points out that no matter how sophisticated 
learning mechanisms might be, they crucially depend 
on the input they get and the feedback they receive—as 
the saying goes in contexts that deal with data: garbage 
in, garbage out. He thus maintains his rather pessimis- 
tic view and concludes that moral intuitions might offer 
reasonable advice in our everyday social life, such as 
coordinating social interactions within a group (“Me 
versus Us,” as Greene puts it), but are likely to fail when 
it comes to solving problems between groups with com- 
peting interests (“Us versus Them”), the moral issues sur- 
rounding novel or unfamiliar problems that originate in 
distinctively modern institutions, practices or technolo- 
gies, or making progress in moral philosophy. 

What can we conclude from these findings and reflec- 
tions? On the one hand, the worry that people’s moral 
judgments are not rational because they are based on 
intuitions seems misguided. Intuitions can be the output 
of highly sophisticated learning systems, and they prob- 
ably have the potential to reflect moral truths and to 
successfully navigate us through, and solve problems in, 
the social world. On the other hand, there is no moral 
safeguard in these systems ensuring that only morally 
relevant information is picked up and that only morally 
adequate outputs are produced. Hence, the prospects of 
the rational-learning strategy for vindicating people’s 
moral intuitions seem to depend on the extent to which 
the sophisticated internal learning mechanisms shap- 
ing our moral intuitions can avail themselves of morally 
relevant information in the external social environment 
(Gjesdal, 2018; Kumar, 2016). Investigating the condi- 
tions under which this endeavor can succeed will prob- 
ably (and hopefully) be a major focus of future moral 


psychology. 
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Notes 


1. To avoid misunderstandings, we do not mean to imply that 
the studies presented in the normative part were designed with 
the purpose of answering normative questions (although some 
may have been). Rather, most of them can be characterized 
as descriptive work that has more less direct implications for 
normative issues. 


2. Unfortunately, the labels “consequentialist” and “deontolog- 
ical” are often used in a crude and loose way in moral psychol- 
ogy (e.g., intervening in Switch is actually consistent with most 
deontological moral theories). See Sinnott-Armstrong (2019) 
and Alexander and Moore (2016) for a comprehensive review of 
consequentialist and deontological ethics, respectively. 


3. Crockett (2013) explains such means-—versus-side effect 
cases by postulating an interaction of the model-based system 
and a third, Pavlovian system, which responds reflexively to 
aversive and rewarding states. 


4. Railton (2017) reviews research specifically on moral learn- 
ing. See also a recent special issue of Cognition on moral learn- 
ing (Cushman, Kumar, & Railton, 2017). 


5. Fehige and Wessels (see chapter 12.1 in this handbook) dis- 
cuss at greater length the relationship between morality and an 
instrumental view of rationality and conclude that they cannot 
be brought into full harmony. 
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Section 13 Visual and Spatial Reasoning 


13.1 Logical Reasoning with Diagrams 
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Summary 


This chapter examines the role of diagrams in reasoning. 
It exposes the tension between formal and informal use 
of diagrams. Diagrams are a complementary represen- 
tation to symbolic formulae. Since choosing the right 
representation may be essential to solving a problem, we 
study diagrams as one such suitable choice. We analyze 
algorithmic and computational approaches to formal- 
izing the logic of diagrams and present some modern 
implementations of diagrammatic reasoning systems. 


1. Diagrams through Time 


Humans have used diagrams to convey reasoning since 
the ancient times of Aristotle and Euclid. In fact, until the 
end of the 19th century, diagrams were considered a legit- 
imate formal tool for explaining the rationale of the solu- 
tion to a problem. They were a solution in themselves. But 
while Euler (1768), Venn (1881), and Peirce (1933) were 
devising logics for their diagrams, Hilbert, Gddel, Russell, 
and Wittgenstein formalized symbolic logic. This formal 
movement cemented the notion that only sentences that 
can be deduced from core axioms through a process of 
inference are considered a proof of a theorem: 


A theorem is only proved when the proof is completely inde- 
pendent of the diagram. The proof must call step by step on 
the preceding axioms. The making of figures is [equivalent 
to] the experimentation of the physicist, and experimental 
geometry is already over with [laying down of the] axioms. 
(Hilbert, 1894/2004) 


Diagrams, in any form, have until the rise of symbolic 
logic not been formalized in this Hilbertian way: there 
were no axiomatic diagrams,' there were no formalized 
inference steps of diagrammatic manipulations, and 
moreover, mathematics was full of erroneous solutions 
due to faulty or deceptive diagrams.” Yet mathemati- 
cians continued to use diagrams to aid their reasoning 
and to convey the intuition of a solution. 


The prejudice against the formal status of diagrams 
and visual methods of reasoning with them was finally 
overturned by Shin’s seminal work on the formalization 
of Venn diagrams (Shin, 1994). Shin devised a formal 
syntax and semantics of particular forms of Venn dia- 
grams, and proved her logic sound and complete. This 
legitimized and spurred on renewed interest in studying 
and devising systems for reasoning with diagrams, some 
of which will be surveyed here. 

This chapter examines the role that diagrams play 
in human reasoning from the computational, algorith- 
mic, and implementational points of view.’ We argue that 
diagrams can encode the part of human reasoning that 
carries the intuition of the solution to a problem and 
can, moreover, convey the proof without needing to be 
supplemented by symbolic logic. Human rationality and 
reasoning are intimately related, as the reasoning pro- 
cess provides the rationale for a human understanding 
of, and belief in, the correctness of a solution. Our inter- 
est does not lie in machine-oriented approaches to rea- 
soning, for which symbolic logic is typically used (with 
perhaps a small exception in the case of the motivation 
for natural deduction). Instead, we study diagrammatic 
reasoning and systems as one plausible way to model 
human-oriented approaches to rationales and therefore 
reasoning. Machine-oriented approaches to reasoning 
are typically motivated by the search for a categorical 
answer to the question of whether a conjecture is a the- 
orem, by the speed of finding a proof, or by the num- 
ber of proofs that the system is able to find. By contrast, 
human-oriented approaches to reasoning seek solution 
explanations that are understandable to humans. This 
human-oriented approach to modeling reasoning is moti- 
vated by the goal of general artificial intelligence (AI) to 
make machines more human-like in their reasoning and 
in the solutions that they find.* The hope is that diagram- 
matic reasoning systems will give us insights into human 
reasoning, which can then be modeled on machines to 
make them more human-like. As AI systems become ever 
more ubiquitous in our interactions with the world, it 
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is important to build systems that humans find amena- 
ble and understandable. The new legal requirement for 
explainability makes the combination of sound reason- 
ing and accessibility very timely. In part, this means cap- 
turing the human intuition and rationale of a solution in 
some way—the diagram presents not only a solution but 
also an explanation of why the solution is a correct one. 
We aim to demonstrate that this human-like reasoning 
with diagrams can be computationally modeled in logi- 
cally formal ways on machines. 


2. Representations 


The importance of problem representation has been 
acknowledged by many researchers (Amarel, 1968; Simon, 
1996). Polya wisely pointed out in his books How to Solve 
It (1957) and Mathematical Discovery (1965) that reason- 
ing about a problem depends on how one represents the 
problem in the first place. J. R. Anderson’s (1978) mim- 
icry argument adds the inseparability of the format of 
a representation and the processes that operate on this 
representation. This is even more immediately apparent 
when modeling reasoning with diagrams: representing 
the problem in the right way may lead to a trivial solution 
that captures the intuition and truthfulness in an obvi- 
ous and accessible way. Representing the problem inad- 
equately may prevent us from ever finding a solution. In 
contrast to machines, which typically use a single fixed 
representation for a problem, most humans are good at 
choosing the representation that is just right to enable 
them to solve a new problem. Moreover, given the right 
representation, human problem solving is dramatically 
improved, and humans recognize the benefits of such 
changes (Cheng, Lowe, & Scaife, 2001). Take, for exam- 
ple, the famous problem of the mutilated checkerboard in 
figure 13.1.1. It is called “mutilated” because two diago- 
nal corners have been removed. The question is, Can you 
cover this mutilated checkerboard with dominoes? 

One could try many different ways to tile the board to 
find out if it can be covered. However, if we change the 
representation and color the board black and white like 
a chessboard (figure 13.1.2), then it is immediately obvi- 
ous that the mutilated checkerboard cannot be covered 
with dominoes, since by mutilating it, we have removed 
two squares of the same color. But we need the same 
number of black and white squares since each domino 
covers one black and one white square. This is obvious 
for humans, but how would a machine go about finding 
this helpful representation? 

It is difficult to see how to use Pdlya’s advice about 
representations in computational reasoning systems. 
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Figure 13.1.1 
Mutilated checkerboard. Can it be covered with dominoes? 


Figure 13.1.2 

No, this mutilated checkerboard cannot be covered with domi- 
noes, because it has more black than white squares, but there 
must be the same number of black and white squares since a 
domino covers one of each. 


Unlike humans, machines’ representations are usually 
fixed by their programmer, and while they can be from 
a wide range, they are typically a single symbolic repre- 
sentation. What machines don’t currently do is reason 
about what system of representation to adopt. They 
deal in local parameters of whatever representation 
they are programmed in. Enabling a reasoning system 
to choose the right representation in problem solving is 
a big open challenge that has only recently started to be 
addressed (Raggi et al., 2020). Its importance lies in the 
relation between the problem representation and the 
computational complexity of finding its solution: imag- 
ine symbolically describing and solving the mutilated 
checkerboard example, and contrast this with the dia- 
grammatic solution in figure 13.1.2. Larkin and Simon 
(1987) extensively discussed the importance of diagrams 
in many domains of reasoning: they claim that “a dia- 
gram is (sometimes) worth ten thousand words.” Many 
so-called diagrammatic proofs without words can be 
found in Nelsen’s books Proofs without Words: Exercises in 
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Visual Thinking (1993) and Proofs without Words II: Exer- 
cises in Visual Thinking (2001). 

There is strong evidence from cognitive psychology 
that diagrams have many properties that often enable 
humans to process them with immediate comprehen- 
sion. Johnson-Laird (1983) and Hegarty and Just (1993) 
argue that humans, at least in some cases, use diagrams 
in their mental models of a situation. It is important to 
recognize that the wrong level of granularity or too much 
visual clutter can impede cognitive processes (Alqadah, 
Stapleton, Howse, & Chapman, 2014; Knauff, 2013), but 
if diagrams capture the logical structure of the problem, 
they can be helpful in reasoning and problem solving 
(see chapter 13.3 by Knauff, this handbook). Diagrams 
concisely store information, explicitly represent the rela- 
tions among the elements of the diagram, and support a 
lot of perceptual inferences that are very easy for humans. 
Shimojima (1996), for example, characterized a property 
of diagrams called “free rides”: expressing some chosen 
information in a diagram often results in expressing other 
pieces of information that are consequences of the chosen 
information; informally, these other pieces of information 
are free rides. Figure 13.1.3b gives an example of how a 
Euler diagram representation of expressions in symbolic 
logic in figure 13.1.3a gives rise to a free ride about the 
fact that R and Q are disjoint: this can directly be read 
off from the diagrams in figure 13.1.3b, whereas it must 
be inferred from the symbolic sentences in figure 13.1.3a. 
Notions such as free rides give one possible explanation of 
why humans find diagrams so accessible. 

Modeling this inherently human ability to choose or 
change appropriate representations in computational 
systems is an open research question. To address it, we 
need to find out which cognitive processes humans use 
to select representations, what criteria they use to choose 
them, and how we can model this ability on machines. 
The choice of representation, whether symbolic or dia- 
grammatic (or even among a number of different sym- 
bolic or diagrammatic representations), will depend on 


(PN Q)=6 
RCP 


(a) Set-theoretic symbolic formulae. (b) Euler diagram. 


Figure 13.1.3 

Free ride: in Euler diagrams, the fact that R is disjoint from Q 
can be read off, whereas this has to be inferred from the sym- 
bolic formulae. 
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whether the target audience is a human (and on their level 
of expertise) or a machine. Such a computational system 
could then readily assess the human and capitalize on the 
explanatory power of diagrammatic representations. 


3. Logical Systems of Diagrams 


Despite the persistent mistrust in the logical status of 
diagrams, the end of the 20th century started to see a 
redressing of this issue (M. Anderson, Meyer, & Olivier, 
2001; Chandrasekaran, Glasgow, & Narayanan, 1995). 
Examples include formalized logical systems of diagrams 
(Hammer, 1995; Howse, Stapleton, & Taylor, 2005; Shin, 
1994). This directly abolished the widely held Hilbertian 
theoretical objections to diagrams being used in proofs. 
Some of this seminal work is presented here. 

While Peirce (1933) began to formalize a variant of 
Venn diagrams and diagrammatic inference rules on 
them, it was Shin (1994) who was the first to show 
that two particular systems of Venn diagrams can have 
a formal syntax, semantics, and model theory. A Venn 
diagram (Venn, 1881) expresses all possible logical rela- 
tions between a fixed finite number of sets. It does so 
by depicting sets as regions inside closed curves, which 
overlap in all possible ways (see figure 13.1.4). Venn dia- 
grams use shading to represent empty sets. 

Shin (1994) formalized a form of Venn diagrams, called 
“Venn-I,” where she used shading to express set-emptiness 
(as in the original diagrams by Venn), x-sequences to 
express that a set is not empty, and linked x-sequences 
to express disjunction (similar to Peirce®). Shin’s further 
Venn-II system introduced connecting lines between dia- 
grams to express disjunctive information. Figure 13.1.5 
shows statements in both Venn-I and Venn-II systems. 

Importantly, Shin equipped both systems with for- 
mal syntax, semantics, as well as sound and complete 
inference rules and showed that Venn-II is expressively 
equivalent to monadic first-order logic. This was the first 


Figure 13.1.4 
Venn diagram representation of the information in figure 13.1.3. 
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Venn-I 


Figure 13.1.5 


Venn-Il 


The Venn-I diagram expresses the information in figure 13.1.4 and also that there is at least one Q that is neither P nor R and there 


is at least one P that is not Q. The Venn-II diagram expresses that either P is a subset of Qand Q\P is not empty, or R\P is not empty. 


time that a diagrammatic system for reasoning received 
a Hilbertian treatment. Shin showed that diagrams in a 
purely diagrammatic reasoning system can be used as 
formal tools. 

Hammer (1995) extended Shin’s work to Euler dia- 
grams: he defined their formal syntax and semantics, 
and proved that a set of diagrammatic inferences were 
sound and complete. Euler diagrams (Euler, 1768) use 
closed curves that can overlap, be totally enclosed by 
other curves, or do not overlap at all, that is, exclude 
each other, to represent intersection, subset, and disjoint 
sets, respectively (see figure 13.1.3b). 

Spider diagrams adapt Hammer's Euler and Shin’s Venn- 
II diagrams. They are based on Euler diagrams, but instead 
of using x-elements to indicate nonempty regions, they 
use “spiders,” that is, dots connected by lines, to repre- 
sent the existence of distinct elements. Spider diagrams 
use shading: in a shaded region, all elements are denoted 
by spiders. So, a shaded region with no spiders represents 
the empty set (see figure 13.1.6). 

The syntax and semantics of spider diagrams were for- 
mally defined in Howse et al. (2005). The language of spi- 
der diagrams is also accompanied by sound and complete 
inference rules, which result in the logic of spider dia- 
grams. This logic is expressively equivalent to monadic 
first-order logic with equality (Stapleton, Howse, Taylor, 
& Thompson, 2004). A set of sound inference rules for 
spider diagrams was defined and proved complete in 
Urbas, Jamnik, and Stapleton (2015). Thus, spider dia- 
grams are more expressive than Venn-II diagrams. 

The revived interest in diagrams has seen other 
domains formalizing a rigorous syntax and semantics of 
diagrams for reasoning. Some are equipped with proved 
formal properties of soundness (which is a minimal 
requirement for using them in proofs) and some even 
with completeness. Examples include analogical reason- 
ing in Grover (Barker-Plummer & Bailin, 1997), diagram- 
matic reasoning about arithmetic in Diamond (Jamnik, 


v 
to 


Figure 13.1.6 

Spider diagram: R is a subset of P and has exactly two elements, 
$s, and $2; Qis disjoint from P (and k); element s3 is either in P or 
in Q, but not in both. 


2001), ontology reasoning with concept diagrams in 
iCon (Shams, Sato, Jamnik, & Stapleton, 2018), and com- 
puter science specification with UML diagrams based on 
higraphs (Rumbaugh, Jacobson, & Booch, 1999). Similar 
to the formalization of Venn, Euler, and spider diagrams, 
all of these systems provide evidence that diagrams can 
be used in formal proofs and thus show that diagrams 
can be made Hilbertian. 


4. Implementations of Diagrammatic 
Reasoning Systems 


In parallel with theoretical interest in the logic of dia- 
grams revived at the end of the 20th century, several 
diagrammatic mechanized reasoning systems have been 
implemented. The motivations for their implementations 
differ: from cognitive ones about improved accessibility 
of reasoning systems to proof of concept that diagrams 
can be formally used for mechanized machine reason- 
ing. They provide one model of reasoning that is more 
human-like than traditional symbolic machine-oriented 
approaches: they are both formal and explainable. 

In implementing diagrammatic reasoning systems, 
there are many design choices that need to be carefully 
considered. For example, how do we devise algorithms 
for laying out diagrams in the best and clearest possible 
way, is the set of inference rules sound and complete, 
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what level of abstraction of inference rules is most acces- 
sible to humans, what is the notion of a diagrammatic 
proof/solution, and should diagrams be general or should 
we exploit their concreteness? Here we demonstrate how 
some of these choices were made for three particular 
implementations of diagrammatic reasoning systems: 
Hyperproof, Diamond, and Speedith. 


4.1 Hyperproof 
Hyperproof (Barker-Plummer, Barwise, & Etchemendy, 
2017; Barwise & Etchemendy, 1991) was the first system 
that implemented the use of both diagrammatic and sym- 
bolic representations to derive conclusions. The diagram 
is a blocks-world situation depicted in a graphical display: 
the upper part of Hyperproof’s screen in figure 13.1.7.° 
In addition, some symbolic formulae of first-order logic 
might be given (in the lower part): Dodec(c) ~ Dodec(d) 
and Small(c). The aim is to show that some conjecture 
about the given information, usually expressed symboli- 
cally, is a consequence or a nonconsequence of the given 
information. In the example in figure 13.1.7, we want to 
determine whether block c and block d are of the same 
shape: SameShape(c,d). Hyperproof combines inspecting 
the diagram and applying logical rules in order to derive 
the conclusion. 

The proofs in Hyperproof are guaranteed to be correct 
if every step, diagrammatic or symbolic, is evaluated to be 
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true according to the evaluation schema in Kleene logic. 
They are heterogeneous since they combine symbolic 
and diagrammatic information. This notion of a het- 
erogeneous proof is novel and stands in contrast to the 
Hilbertian view of formal proof. Using the blocks-world 
situation exploits not only diagrammatic representations 
but also their concreteness property—concrete proof 
objects can be named and their shape and size inspected. 
Hyperproof has been successfully deployed in an educa- 
tional setting to teach first-order logic precisely for these 
properties: they make the reasoning more accessible to 
most users.’ 


4.2 Diamond 

Diamond is the first system that implemented the con- 
struction of purely diagrammatic proofs (Jamnik, 2001). 
Its domain is natural number arithmetic for inductive 
theorems. Uniquely, Diamond exploits the concrete- 
ness property of diagrams to express a particular con- 
crete instance of a theorem and its proof, rather than 
the general, universally quantified case. For example, 
figure 13.1.8 gives a diagrammatic proof for a theorem 
about the sum of odd natural numbers for a concrete 
instance n = 6. The proof cuts a square of size 6 into a 
sequence of so-called ells,* each representing a subse- 
quent odd natural number since it consists of two edges, 
2n, but the joining vertex is counted twice, so 2n-1. In 
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Figure 13.1.7 
Hyperproof’s proof. 
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Figure 13.1.8 
Diagrammatic proof of the theorem about the sum of odd natu- 
ral numbers. 


the concrete case of n = 6, the inference [cut was carried 
out five times. 

Diamond’s approach to capturing the generality of 
the proof while using concrete instances is theoreti- 
cally justified by the constructive w-rule (Baker, Ireland, 
& Smaill, 1992). This rule states that if there is a uni- 
form effective procedure generating every instance of 
a proof, then we can conclude the universally quanti- 
fied theorem. One such effective procedure is a recursive 
program. Therefore, Diamond’s general proof is encoded 
in the recursive program that, upon an input for every 
value of the universal quantifier, produces a proof for 
that instance of the theorem. 

The work on automation of diagrammatic proofs in 
Diamond provides important information on proof pro- 
cedure construction. It demonstrates the importance of 
representing diagrammatic expressions so that general 
reasoning techniques can be applied to them. Further- 
more, it provides insight into how diagrams and purely 
diagrammatic inferences can be used in formal proofs. 


4.3 Speedith 
Speedith (Urbas et al., 2015) is an interactive diagram- 
matic theorem prover for spider diagrams. It is the first 
implemented system with the same notion of proof as in 
a typical symbolic theorem prover: apply inference rules 
until tautology or contradiction is reached. Instead of 
symbolic formulae, the theorem is expressed using dia- 
grams. The inference rules are diagrammatic (and some 
symbolic) and form a sound and complete set—this 
guarantees that diagrammatic proofs are correct. 

Figure 13.1.9 shows an example of a diagrammatic 
proof as constructed in Speedith. Here, the antecedent 
of the Initial proof goal is a spider diagram that conveys 
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information about the relationships between two ele- 
ments and two sets, and it proves that the consequent 
of the Initial proof goal follows logically. In particular, the 
proof establishes that given sets A and B, if there are two 
elements, one of which is in both A and B and the other 
is either only in A or only in B, then we can deduce that 
one element is in A and the other is in B. 


5. Discussion 


The rationality of human thought has been studied from 
a number of different angles. Cognitive, psychological, 
and philosophical perspectives include investigations 
into mental models, heuristics, deduction, argumenta- 
tion, beliefs, social epistemology, morality, and learning, 
among others. This chapter presented an angle stemming 
from the perhaps less mainstream, yet foundational, arti- 
ficial intelligence view of one aspect of human rationality, 
namely, that of human-oriented visual mathematical rea- 
soning. We approached rationality from the algorithmic 
and implementational view to computationally model 
reasoning with diagrams. The invention of Hilbertian rea- 
soning at the start of the 20th century firmly rooted the 
formality of human thought in symbolic logic. This was 
despite centuries of mathematicians using visual tools 
such as diagrams to convey theorems and their proofs. 
The human ability to “see” the problem and its solution 
in a diagram is one of the fundamental components of the 
human mathematical cognitive repertoire. Nevertheless, 
there is a clear tension between symbolic and diagram- 
matic representations when computationally modeling 
human intuition in problem solving. 

Unlike sentential, symbolic, or linguistic represen- 
tations, which represent relations between objects in 
a single dimension by concatenation, diagrammatic 
representations use multidimensional relations between 
objects. This has both advantages and drawbacks. On 
the plus side, once the relations between objects are cor- 
rectly represented, their properties or new relations can 
be just read off from the diagram. Indeed, diagrams may 
give you new information for free that was not originally 
declared and represented—referred to as “free rides.” On 
the negative side, some relations cannot be expressed in 
diagrammatic representations by utilizing spatial prop- 
erties, and if relations are not represented completely, 
this may lead to unintended and erroneous conclusions. 
Stenning and Oberlander (1995) argue that diagrams 
are necessarily concrete, with just a limited capacity 
for abstraction. Knauff discusses this tension between 
concreteness and abstraction, and the level of granular- 
ity in diagrams, in chapter 13.3 of this handbook. The 
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Screenshot of a proof of a spider-diagrammatic 


statement as constructed in Speedith. 
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consequence is that the user has to distinguish intended 
from unintended concreteness (e.g., the use of n = 6 
above). Getting this wrong often causes erroneous con- 
clusions. There are numerous examples in the history 
of mathematics of faulty solutions—Lakatos (1976) and 
Maxwell (1959) illustrate some better-known ones. 

When are diagrammatic representations not expres- 
sive enough? The systems we mentioned here are all 
limited in expressivity in different ways. For example, 
Venn-I cannot express disjunctive information, spider 
diagrams can only express monadic first-order logic with 
equality, and Diamond can only tackle arithmetic theo- 
rems expressed as three-dimensional shapes. Diagrams 
are concrete objects and are thus subject to spatial con- 
straints that sentential representations do not have. This 
is clearly limiting, and there are sentential logics, and 
their computational implementations in theorem prov- 
ers, that are more expressive and indeed very impressive 
(e.g., Coq’s proof of the famous four-color theorem; Gon- 
thier, 2008). 

However, purely sentential logics are limited in repre- 
sentational scope and often unilluminating with regard 
to solution explanations. The diagrammatic approaches 
to reasoning presented in this chapter provide a vehicle 
for widening the scope and expanding the explanatory 
power of computational systems for reasoning. They 
work with users in accessible and explanatory ways and 
can be formal in the Hilbertian sense. They enable us to 
explore the importance of representational choice on the 
diversity of inference methods and the ability to actually 
solve problems in mathematics. Ultimately, the goal is to 
understand human rationality and, from the AI point of 
view, to build computational systems that reflect it—this 
is going to make AI machines more human-like in inter- 
nal workings and in external interactions with humans. 


Notes 


1. The distinction between symbolic and diagrammatic meth- 
ods is not straightforward. While Newton’s famous Philosophice 
Naturalis Principia Mathematica (Mathematical Principles of Natural 
Philosophy, 1726/2020) is resolutely based on geometric tech- 
niques, its contributions are little understood for their diagram- 
matic beauty—they are usually demonstrated and analyzed 
using a symbolic calculus into which they were translated very 
early on. 


2. One of the best-known examples of erroneous proofs is 
Cauchy’s visual proof of Euler’s theorem, which is well docu- 
mented in Lakatos’s (1976) seminal work on the analysis of 
mathematical discovery. This “proof” stood for over 50 years 
before a Hilbertian symbolic formalization was devised that 
could readily be checked for errors. 
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3. The computational, algorithmic, and implementational views 
of rationality are defined in the Introduction to this volume by 
Knauff and Spohn. Here, our implementational view does not 
refer to the hardware that embodies rationality but rather to the 
software that executes reasoning tasks. 


4. Current mainstream AI is mostly interested in machine- 
oriented statistical learning. Nevertheless, modeling human- 
like reasoning on machines remains one of the foundational 
motivations of the AI field. 


5. Peirce did not use shading for set emptiness, but he intro- 
duced x (for nonemptiness) and o (for emptiness) elements 
that can be linked with lines to express disjunctive informa- 
tion in the diagrams. These diagrams, however, lost somewhat 
the spatial appeal and ease of reading that Venn diagrams 
have. 


6. The diamond in the lower part of the display denotes the 
blocks-world situation—clicking the diamond displays the situ- 
ation graphically in the upper part. 


7. Stenning and Oberlander (1995) showed that there are indi- 
vidual differences in users’ performance: users who are more 
visual reasoners do better with visual representations, and 
sentential reasoners do better with sentential representations. 


8. Two perpendicularly joined sides of a square look like a 
letter L, so they are referred to as ells. Splitting an ell from a 
square is thus referred to as an Icut. 
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13.2 Rational Reasoning about Spatial and Temporal Relations 
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Summary 


Reasoning about spatial and temporal relations is ubiq- 
uitous in everyday life. It is important that such rea- 
soning processes proceed rationally; otherwise, we will 
arrive at the wrong place or appear there at the wrong 
time. To demonstrate the specifics of the underlying 
cognitive processes, human spatial and temporal rea- 
soning is compared to formal approaches from artificial 
intelligence (AI). The underlying cognitive processes 
have particular characteristics: First, they are qualitative, 
that is, metrical information is taken into account only 
when necessary. Second, they realize resource-bounded 
rationality, that is, due to limited resources, only par- 
simonious mental representations are built: often a 
preferred mental model is constructed and, based on it, 
putative conclusions are drawn. Third, they are sensi- 
tive to the mental effort problems require, that is, often 
mental models are constructed that require a small 
number of operations only. This often leads to rational 
spatial and temporal inferences but can also lead to sys- 
tematic errors. 


1. Space and Time Are Fundamental in Everyday Life 


Humans are always within the realm of space and 
time. But we are not always consciously aware of both 
dimensions of our existence. Kant (1781/1787/1995, 
A23/B38) claimed that space and time are not empiri- 
cal (in the sense that “abstract” space or time is not 
perceivable by our senses) but rather “a priori represen- 
tations.” We also cannot perceive abstract space and 
time directly but become aware of the reality of space 
and time whenever we navigate, communicate, or rea- 
son about our environment or about events in time. In 
the following, we start by considering a simple spatial 
reasoning problem concerning four entities and con- 
sisting of two pieces of relational information. Please 
read the following information and try to answer the 
question afterward. 


You are now at the train station. 


Your bicycle is in front of the north entrance of the 
church. 


The church is north of the train station. 


What, if anything, can be inferred from the given 
information? 


An immediate conclusion, that is, an inference that can 
easily be drawn, is that you have to head north to find 
your bicycle. Analogously, let us assume that you know 
that 


The complete decoding of the human genome took 
place after the first moon landing. 


The invention of the computer took place before the 
first moon landing. 


It is easy for you to infer from these two statements that 
the computer was invented before the complete decod- 
ing of the human genome took place. With some back- 
ground knowledge, such as that computers are nowadays 
often involved in scientific progress, you can even form a 
putative conclusion that the invention of the computer 
was very likely important in the complete decoding of 
the human genome. 

These two examples are similar to a multitude of 
other problems we encounter in everyday life. Let us 
consider some characteristics of such problems: first, the 
relations are qualitative, that is, no numerical informa- 
tion is given, and you do not need any numerical infor- 
mation or may have even thought about it. Second, all 
information is relevant to solving the problem. Third, in 
our example, the four terms in the spatial problem and 
the three events in the temporal problem were relatively 
simple, but it is not hard to imagine that additional 
information may make such problems more difficult to 
solve. 

In solving such problems, you may have had the 
impression that you formed a vivid mental picture or 
that you used an abstract relational representation of the 
described spatial situation. Or, instead, you may have 
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applied logical inference rules (e.g., a transitivity rule) 
to the two relations “in front of (the north entrance of)” 
and “north of” (despite being nonidentical, they do not 
differ in their direction), without forming an analogous 
model-like representation, to infer that you have to head 
north. Or you may have applied a heuristic to generate a 
conclusion. An example of such a heuristic might be to 
use the only cardinal direction relation (“north”) men- 
tioned in the given information, giving you the gen- 
eral direction in your conclusion. While for this sample 
problem, each type of inference process can be applied, 
all leading to the same answer, it is very unlikely that 
a human reasoner employs all of these processes. It is 
much more plausible that there are specific processes 
that most reasoners tend to use. Probably an important 
observation is that humans often use qualitative spatial 
and temporal relations, which do not represent exact, 
metrical information in a continuous way. Rather, they 
only represent those relations that are necessary for the 
inference. This allows humans to represent spatial and 
temporal information even if exact information is not 
available or would require too much effort to acquire. 
An advantage of qualitative relations is that they do 
not require any artificial external metrical scale (Freksa, 
1992). Therefore, qualitative spatial and temporal rela- 
tions are more practical in an environment where exact 
information is missing. A qualitative relation can be con- 
sidered an “abstraction summarizing similar quantita- 
tive states into one qualitative characterization” (Moratz 
& Ragni, 2008, p. 76; see also Rauh et al., 2005). In this 
sense, qualitative descriptions are compressed represen- 
tations and are much more useful in a complex environ- 
ment for an agent that has a bounded rationality only. 

An underlying assumption of this chapter is that the 
way in which human reasoning proceeds depends on 
the need to be applicable and useful in a complex world, 
given that cognitive resources, such as working memory, 
are limited. We will consider the underlying processes of 
reasoning about spatial and temporal beliefs both from 
a normative perspective (i.e., looking at logical formal- 
isms capturing idealized knowledge representation and 
reasoning) and from a descriptive perspective (i.e., how 
humans actually represent knowledge and reason about 
it). By contrasting what ought to be and what is, it is pos- 
sible to identify specific differences between formal and 
human reasoning. The chapter addresses this question 
on a formal level, by introducing artificial intelligence 
(AI) systems based on qualitative relations; on an empiri- 
cal level, to gain access to human reasoning; and on an 
algorithmic level, by explaining the cognitive processes 
via cognitive modeling. 


Marco Ragni 


2. Formal Models for Reasoning about Spatial 
and Temporal Relations 


Since it is natural for humans to draw inferences such 
as the ones above, we often consider these problems as 
simple. From a formal perspective, however, they are cer- 
tainly not. In the past years, cognitively inspired calculi 
(e.g., the double-cross calculus; Zimmermann & Freksa, 
1996) have been introduced that help us to reconstruct 
how the example problems can be solved. Two famous 
calculi are the interval calculus introduced by Allen 
(1983) and the region connection calculus introduced 
by Randell, Cui, and Cohn (1992). 

The interval calculus was initially developed for 
reasoning with temporal relations but was soon trans- 
ferred to spatial reasoning (Guesgen, 1989). In the spa- 
tial domain, the calculus works with solid objects. The 
calculus can also be extended into more complex for- 
malisms (Ligozat, Mitra, & Condotta, 2004). An event 
can be represented by its start time and its end time (see 
table 13.2.1). Accordingly, two events can be qualita- 
tively related by the relations between the start and end 
times of the two events, for example, that the first event 
ends before the second starts, or that the first event over- 
laps with the second, or that both events start at the 
same time, or that both finish at the same time, or that 
the first event takes place during the second event. If 
the qualitative relations between start and end points 
are known, then 13 different relations can be identified 
(see table 13.2.1). Imprecise knowledge about the start 
and end times of the events is represented by disjunc- 
tions of these 13 relations (e.g., an event finishes before 
the other starts or they overlap). 

The region connection calculus (RCC) has been devel- 
oped in the spatial domain and is similar to the inter- 
val calculus, but instead of orientational information, it 
just uses topological information, that is, the relations 
between the boundaries of objects (e.g., two regions can 
be disjoint, or they can overlap, or the first region can be 
a proper part of the second, or vice versa, or they can 
be identical). The calculus using these five basic topo- 
logical relations is called RCC-S. If the boundaries of the 
regions are relevant too (e.g., when two countries share 
a common border), then three more relations can be 
formed: a region touches another from the outside or 
they are disjoint (and not even their borders overlap), 
or the first region is a proper part of the second and 
touches the latter’s border or not, or this relation holds 
for the second region. With the resulting eight relations, 
we have the calculus RCC-8 (see figure 13.2.1). Based on 
these base relations, one can again express indeterminate 
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The 13 qualitative interval relations of the interval calculus, their names for temporal rea- 


soning, natural language expressions for the spatial domain, and an interval representation 


Temporal Spatial Relation Inverse Pictorial Representation 
before I lies to the left of J IbJ J bil I ——] 
meets I touches J at the left ImJ Jmil =, 
overlaps T overlaps J from the left JIoJ Joil — J 
during Tis completely in J IdJ J dil =i J 
starts I lies left-justified in J IsJ Jsil =i) 
finishes I lies right-justified in J If] J fil — j 
equals I equals J IeJ Jel —; 
Note: Adapted from Knauff (1999). 

XDC Y XPO Y X TPP Y X NTPP Y 

Figure 13.2.1 
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XEC Y XEQY X TPPi Y 


descriptions reflecting imprecise knowledge (e.g., that a 
region contains the other or that they are disjoint). 
These are just two of many approaches that have been 
suggested in AI for reasoning in a rational way with spa- 
tial and temporal relations. These approaches can serve as 
normative accounts of how correct reasoning with quali- 
tative spatial and temporal relations should proceed. They 
have many interesting formal characteristics and have 
been used in many domains of application, for example, 
in robotics (Moratz & Ragni, 2008), plan generation (e.g., 
Westphal, Dornhege, WoOlfl, Gissler, & Nebel, 2011), sea 
navigation (e.g., Wolter et al., 2008), and language pro- 
cessing (e.g., Mast, Wolter, Klippel, Wallgriin, & Tenbrink, 
2014). These calculi are of interest in this chapter also 
because many cognitive psychologists have used them to 
compare human reasoning with reasoning in technical sys- 
tems (Knauff, Rauh, Schlieder, & Strube, 1998; Rauh et al., 


The region connection calculus with eight base 
relations (RCC-8). X and Y are regions. The eight 
base relations are DC = disconnected, PO = par- 
tially overlaps, (N)TPP = (non)tangential proper 
part, EC = externally connected, EQ = equal, and 


XNTPPi Y the two inverse relations NITPi and TPPi. 


2005; Schlieder & Berendt, 1998) and also to determine 
the correct solutions to inference problems. The results of 
these empirical studies show that the formal approaches 
capture some aspects of human reasoning but that people 
also quite often reason with spatial and temporal relations 
differently, and there are cross-cultural similarities as well 
as differences (e.g., Knauff & Ragni, 2011). 

The problem with many formal calculi is that they can 
be computationally expensive (for an overview, see Renz 
& Nebel, 2007), that is, the number of simple compu- 
tations and/or the amount of memory space needed to 
solve the problem rise exponentially with the size of the 
problem description. This makes “large” problems practi- 
cally intractable. A central empirical finding, however, is 
that, in contrast to formal approaches, humans do not 
consider all possible models but rather try to find a use- 
ful specific model, especially when the verbal description 
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is indeterminate (i.e., allows for multiple models). In the 
following, we describe this and related findings from 
empirical research and computational modeling. 


3. Cognitive Findings on Spatial 
and Temporal Reasoning 


Let us first look at the behavioral level by studying which 
conclusions people draw from a set of given premises. 
Such questions have been in the center of reasoning 
research for several decades, and many effects are so reli- 
able that they have been replicated across many studies. 
Here are a few of these robust findings: 


(1) The indeterminacy effect: if a problem description is 
indeterminate, that is, if it allows for multiple sce- 
narios, the problems are on average more difficult 
to reason through. Indeterminate descriptions can 
appear quite often in everyday life (e.g., in cases 
where information is missing). This effect has been 
observed repeatedly and has been explained via 
both mental models and mental logic approaches 
(e.g., Johnson-Laird & Byrne, 1991; Kie&ner & Ragni, 
2017; Van der Henst, 2002). 

(2) Therelational complexity effect: ifa problem description 
contains relations of a higher arity (e.g., “between,” 
which is a ternary relation), or the relational infor- 
mation needs to be decomposed into facts involv- 
ing simpler relations, then the reasoning difficulty 
increases. Hence, relational complexity influences 
the mental representation that is constructed (e.g., 
Goodwin & Johnson-Laird, 2005; Halford, Wilson, & 
Phillips, 1998; Jahn, Knauff, & Johnson-Laird, 2007). 

(3) The visual impedance effect: if a problem descrip- 
tion uses relations that are easier to visualize than 
more abstract relations, then this visualizability can 
impede reasoning (e.g., Knauff, 2013). This indicates 
that the mental representation most human reason- 
ers use is rather abstract and relational and less a 
visual image (e.g., Pylyshyn, 2002). 

(4) The continuity effect. Consider the following two 
descriptions: 

(continuous): A is left of B; B is left of C; C is left of D. 
(discontinuous): A is left of B; C is left of D; B is left 

of C. 

In the first description (continuous), each new 
premise connects directly to the previous one, such 
that the new information can be easily mentally 
integrated into a new representation. In the second 
description (discontinuous), the information cannot 
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be directly related. The first type of description is 
easier for reasoners to process than the second (e.g., 
Knauff et al., 1998; Nejasmic, Bucher, & Knauff, 
2015). Since the information given is identical, an 
explanation for the difference is that for the con- 
tinuous problem, there is an immediate succes- 
sive integration of the premise information into a 
mental model, while in the discontinuous case, the 
processing requires to first store two independent 
pieces of information. 

(5) The relevance of information effect: if reasoners receive 
information that is partially irrelevant for the con- 
clusion (e.g., because the conclusion is already 
given), then such problems are easier than problems 
with the same amount of information where all of it 
is relevant (e.g., Schaeken, Girotto, & Johnson-Laird, 
1998; Van der Henst, 1999). This indicates that spa- 
tial and temporal reasoning difficulty depends on 
the amount of relevant information, not on the 
total amount of information, and that the model 
contains only the relevant information. 


These five effects (for an overview, see, e.g., Ragni, 
2008) have been repeatedly reported and support the 
hypothesis that spatial beliefs are represented by mental 
models based on the principle of minimal operations 
(see below). We will later see that an explanation for 
how humans draw inferences is that often, instead of 
constructing all possible models, they construct just one 
specific mental model, the “preferred” model. This does 
not only reduce the memory load incurred by storing 
models but also reduces the mental effort that comes 
with constructing models. This resource awareness is a 
realization of bounded rationality. Bounded rationality 
is an adaptation of our mind to these limitations, for 
example, of the available information and of our cog- 
nitive resources (see Gigerenzer & Selten, 2002; Simon, 
1955). Hence, another robust result is the following: 


(6) The preference effect. Consider the following example 
from Ragni and Knauff (2013): 
The Ferrari is parked to the left of the Porsche. 
The Beetle is parked to the right of the Porsche. 
The Porsche is parked to the left of the Hummer. 
The Hummer is parked to the left of the Dodge. 


The description allows for the following three 
models: 
Model 1: Ferrari Porsche Beetle Hummer Dodge 


Model 2: Ferrari Porsche Hummer Beetle Dodge 
Model 3: Ferrari Porsche Hummer Dodge Beetle 
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While logically, all three arrangements are models 
of the situation described in the premises, experimen- 
tal studies showed that human reasoners accept model 
1 (which has been called the preferred mental model) 
more often and faster than model 2, which they in turn 
accept more often and faster than model 3 (which are 
called alternative mental models). To make logically 
correct inferences, the construction of all alternative 
models is relevant, because an alternative to the pre- 
ferred model can be a counterexample, which would 
make the inference based on the preferred model 
wrong. For example, the previous premises might be 
presented to reasoners, and then they must answer the 
question, “Is the Beetle (necessarily) parked to the left 
of the Dodge?” In this case, most people do construct 
the first model but not the other two models. Hence, 
they give a wrong answer. 


The preference effect has been reported for the first 
time in Knauff, Rauh, and Schlieder (1995). These authors 
also suggested the term “preferred mental model” to 
explain the cognitive foundations of the preference 
effect. Since then, the preference effect was repeatedly 
found in many experiments and has been replicated 
several times (e.g., Knauff, 2013; Knauff et al., 1998; 
Ragni & Knauff, 2013; Rauh et al., 2005; Schultheis, 
Bertel, & Barkowsky, 2014). We return to this in the 
next section. 


4. Neural Findings and Their Impact for a Theory 
of Spatial and Temporal Reasoning 


The way our brain processes spatial and temporal infor- 
mation can explain how and why we deviate from formal 
approaches to reasoning in AI. For instance, we know 
that, in contrast to formal models of computation, like 
Turing machines, the human mind is not monolithic, 
that is, there is no single storage space; rather, for differ- 
ent tasks, different regions are relevant. Core regions (cf. 
chapter 1.4 by Goel, this handbook) identified for spatial 
and temporal reasoning are the prefrontal cortex (espe- 
cially the dorsolateral and the left prefrontal cortex), 
mainly implementing control functions (e.g., the inte- 
gration of different premises; Fangmeier, Knauff, Ruff, 
& Sloutsky, 2006), and the parietal cortex, mainly stor- 
ing the mental model (Crone et al., 2009; Knauff, 2006; 
Ragni, Franzmeier, Wenczel, & Maier, 2014). A recent 
meta-analysis (Wertheim & Ragni, 2018) of functional 
magnetic resonance imaging studies was able to identify 
the involvement of two frontal areas (Brodmann areas 6 
and 45, with a role in complex planning and semantic 
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understanding, respectively) and one parietal area (Brod- 
mann area 7). For spatial reasoning, especially the supe- 
rior parietal lobe (SPL) is relevant. 

The theory of preferred mental models (Ragni & 
Knauff, 2013) claims that different phases in spatial 
reasoning exist: first constructing the preferred mental 
model and then searching for counterexamples by con- 
structing alternative mental models. Now, a study shows 
that disturbing the information processing in SPL by tran- 
scranial magnetic stimulation (TMS) leads to a decrease 
in people’s ability to construct the preferred mental 
model, but the stimulation of the SPL has no impact 
on the generation of alternative models, that is, mod- 
els 2 and 3 above (Ragni, Franzmeier, Maier, & Knauff, 
2016). This supports that these processes are different, 
because they are realized differently on the neural level. 
Additionally, the study supports the claim that bounded 
rationality is a relevant factor. Reasoners with a smaller 
working memory capacity made more errors in accept- 
ing the preferred and alternative models, which are cor- 
rect models. Reasoners with a smaller working memory 
capacity are more susceptible to disturbance effects by 
TMS and subsequently accept the preferred model less 
often. In fact, the disturbance effect by application of 
TMS was inversely correlated with working memory 
capacity: it was lower for participants with greater work- 
ing memory capacity and higher for those with smaller 
working memory capacity. 

The results show that reasoning about spatial and tem- 
poral relations depends on the preferred mental models 
that are formed in the parietal cortex and on control pro- 
cesses that are relevant for integrating new information 
and are located in the frontal regions. Finally, limited 
working memory capacity can explain why not all mental 
models are constructed. 


5. Computational Models and the Relevance 
of Complexity for Rationality 


We now turn to cognitive computational systems that 
aim to model, reconstruct, and explain the processes 
underlying human reasoning. Some of these cognitive 
computational systems for spatial reasoning use Carte- 
sian coordinate systems (Johnson-Laird & Byrne, 1991; 
Schultheis & Barkowsky, 2011), whereas others use dis- 
crete representational structures such as lists or grids (e.g., 
Krumnack, Bucher, Nejasmic, Nebel, & Knauff, 2011; 
PRISM, Ragni & Knauff, 2013). Some systems assume the 
construction of mental models while others rely on visual 
images (e.g., Casimir; Schultheis & Barkowsky, 2011). Yet 
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others use abstract relational information (e.g., PRISM), or 
are inspired by the linguistic task of encoding and reen- 
coding spatial relations (Krumnack et al., 2011). 

Those computational models that successfully solve 
problems like humans do shed some light on the underlying 
cognitive processes. For instance, most cognitive models 
assume that human reasoners construct mental mod- 
els and make transitive inferences based on these models. 
For the visual impedance effect, current cognitive models 
assume that people initially create a visual mental image 
but then have to extract the relevant spatial information 
from this image, which can be time-consuming and error- 
prone (Knauff, 2013). Others try to explain the effect by 
assuming that the visual relations activate additional fea- 
tures in declarative memory (Albrecht, Schultheis, & Fu, 
2015). 

There exist only a few systems for temporal relational 
reasoning, and these are based on mental models. The 
cognitive processes are similar to those in processing 
spatial relations: humans construct preferred mental 
models and then vary them to test putative conclu- 
sions (Johnson-Laird & Byrne, 1991; Juhos, Quelhas, & 
Johnson-Laird, 2012). Kelly and Khemlani (2019) have 
demonstrated that many features that have been identi- 
fied in spatial reasoning also transfer to durative tempo- 
ral relations, for instance, one event happening during 
another event. Examples of features that transfer from 
the spatial to the temporal domain are that the under- 
lying mental models are iconic, that they rely both on 
intuitive and on deliberative processes, and that prob- 
lems eliciting more models have a higher reasoning dif- 
ficulty. Moreover, the consideration of cognitive costs 
can explain why some reasoners are able to mentally 
simulate alternative models that falsify the conclusion 
suggested by the preferred model. 

We already mentioned that the findings from research 
on spatial and temporal reasoning can be explained within 
the conception of bounded rationality. In particular, in 
cases where several models are possible, humans system- 
atically prefer one model while ignoring others (Knauff 
et al., 1998; Ragni & Knauff, 2013; Rauh et al., 2005). The 
explanation is that such preferred mental models are con- 
structed according to principles of operational parsimony: 
the human mind builds the mental model that needs 
the smallest number of cognitive operations. 

An important point that these findings demonstrate 
is that human reasoners do not generate all possible 
mental models of given information with equal ease and 
equal likelihood. And this in turn shows that logical for- 
malisms such as formal qualitative calculi (see section 2 
above), which do not make any difference between 
mental models, are descriptively inadequate. 


Marco Ragni 


The theory of preferred mental models, in contrast, 
assumes that reasoners successively insert each new object 
from the premises into their mental model, filling a slot 
that is not already occupied by another object, such that 
the resulting model still matches the premises. This pro- 
cess has been algorithmically realized in a computational 
model called prism (Ragni & Knauff, 2013), which indeed 
implements bounded rationality and preference effects, 
based on the idea of simple spatial operations such as 
inserting an object into a mental model or changing the 
position of a mental focus. The number of such operations 
needed for solving a problem forms a measure of its cogni- 
tive complexity. By considering the number of operations 
in a spatial working memory representation, this measure 
can explain the preference effect and which alternative 
model is constructed when (Ragni & Knauff, 2013). The 
number of such operations explains which mental model 
is constructed and inspected, as well as which putative 
conclusion is considered in the reasoning process. If the 
number of operations is high, often only a single model is 
constructed (Ragni & Knauff, 2013). Which model is con- 
structed depends on the specific way the information is 
mentally processed (e.g., whether the information is pre- 
sented in a continuous way, as in the continuity effect), on 
background knowledge (e.g., whether we have a preferred 
arrangement of objects, say, of the silverware on a table), 
and on the underlying spatial domain. For example, it is 
easier to mentally insert objects in large-scale scenarios 
(e.g., cardinal directions; Ragni & Becker, 2010) than in 
small-scale ones (e.g., left-right relations among fruits on 
a table; Ragni & Knauff, 2013). 

The preference effect is ubiquitous and has been rep- 
licated for small-scale spaces (Ragni, Fangmeier, Webber, 
& Knauff, 2006; Rauh et al., 2005), for large-scale spaces 
(with respect to the cardinal directions; Ragni & Becker, 
2010; Schultheis & Barkowsky, 2011), for topological rela- 
tions (even cross-culturally; Knauff & Ragni, 2011), and 
also for other reasoning domains (e.g., Johnson-Laird, 
2006). That the preference effect is so prevalent indicates 
that in the domain of spatial and temporal relational rea- 
soning, rationality is bounded not only by working mem- 
ory limitations but also by limitations on the number of 
mental operations one can perform. Of course, the two 
are related, but we will not discuss this here (see Knauff, 
2013). In the following, some further heuristics of human 
spatial and temporal reasoning are described. 


6. Further Characteristics of Human Spatial 
and Temporal Reasoning 


Although many kinds of spatial and temporal relations 
exist, a particularly important kind of such relations are 
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transitive inferences. In such inferences, a conclusion “A 
is north of C” is drawn from the premises “A is north 
of B” and “B is north of C.” When humans try to solve 
such inferences, they often fall prey to illusions even for 
nonspatial relations such as blood-relatedness: many 
reasoners infer from “A is blood-related to B” and “B is 
blood-related to C” that A is blood-related to C. In fact, 
there is a type of counterexample known to almost all 
reasoners: A is the father, B is the daughter, and C is the 
mother. Still, the internal mental representation and the 
cognitive inference system prompt many human reason- 
ers to make this inference (Goodwin & Johnson-Laird, 
2008). Such illusions demonstrate that our internal 
inference mechanism is overeager to apply transitivity. 
And they demonstrate that we internally represent even 
nonspatial information in a way that allows drawing 
transitive inferences based on spatially organized mod- 
els. Many of the following results are from experiments 
with transitive relations. 


6.1 Human Reasoning about Space and Time Employs 
Mental Models and Manifests Preference Effects 

There is broad agreement among current theories and 
computational models that spatial and temporal beliefs 
are represented by mental models (Krumnack et al., 
2011; Ragni & Knauff, 2013; Schultheis & Barkowsky, 
2011; Johnson-Laird & Byrne, 1991), which are analogi- 
cal representations of states of affairs. But there is also 
some disagreement about how much visual informa- 
tion is present in mental models and in what form it 
is represented (e.g., Pylyshyn, 2002). Experimental find- 
ings indicate that our internal representation is rather 
abstract-relational and less visual. In fact, if additional 
visual features must be encoded, an impedance effect 
on reasoning can occur (Knauff, 2013). The under- 
lying representation is again an abstraction comprising 
only the information relevant for the specific task. This 
principle of minimizing cognitive effort holds also for 
cases where multiple models are possible: usually only 
one particular model is constructed, which comprises 
relevant beliefs. Given limited cognitive resources and 
time pressure, this parsimony is rational. In the follow- 
ing, results are reported that support this claim. 


6.2 Human Reasoning Is Influenced by Problem 
Content and Embodiment 

Experimental studies have shown that problem content 
and the reasoner’s background information about the 
given domain can have strong effects on the construc- 
tion of the preferred mental model (e.g., Rauh, 2000). 
Accordingly, the mechanisms for the construction of spa- 
tial representations even extend to other domains (e.g., 
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mathematical cognition). In the spatial-numerical asso- 
ciation of response codes (SNARC) effect, small numbers 
are responded to faster with the left hand, and large num- 
bers show a similar proclivity with regard to right-sided 
responses and presentations (Hubbard, Piazza, Pinel, & 
Dehaene, 2005). This indicates that people internally rep- 
resent numbers in a spatial ordering from left to right by 
ascending size. For relational reasoning, the same effect 
has been identified for the relational information “A is 
left of B” and “B is left of C” in responding with the left 
hand for items further left and with the right hand for 
items further right (Prado, Van der Henst, & Noveck, 
2008). Thus, even for arbitrary relational information, 
our internal mental representation is not independent 
from our body; hence, our “embodiment” matters. 


6.3 Human Rationality Ils Bounded 

Empirical findings demonstrate that human reasoners often 
do not generate all possible models but only a specific one— 
the preferred mental model. Additionally, not all alterna- 
tive models are constructed, only small modifications of 
the preferred model. Findings from neuroscience demon- 
strate that mental processes are not monolithic: alternative 
mental models are generated in other brain regions than 
the preferred mental model. Working memory capacity 
in turn is a limiting factor for the construction of mental 
models. An analysis of core empirical effects demonstrates 
that problems that are more difficult (e.g., the discontinu- 
ous case or multiple-model problems) require more mental 
operations than easier problem descriptions—supporting 
an operational parsimony. Together, working memory limi- 
tations and operational parsimony are core aspects consti- 
tuting bounded rationality in humans. 


7. Conclusion 


We can draw several conclusions from this chapter. 
Probably the most important are that human reasoning 
about spatial and temporal relations deviates from classi- 
cal conceptions of rationality and that it is multifaceted: 
we do not simply apply a few general principles. Beyond 
the limited information capacity of our senses, human 
rationality is bounded in a twofold way: first, our work- 
ing memory capacity is limited, forcing our brains to 
construct and manipulate specific mental models; sec- 
ond, our computing capacity is limited, whence our 
brains cannot exhaustively survey all possibilities—this 
operational parsimony explains many, if not all, of the 
cited effects. For estimating operational effort, there is a 
cognitive complexity measure that is based on the num- 
ber of simple operations and explains especially why 
some models are constructed and others are neglected. 
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While humans have sometimes been considered irra- 
tional, they are actually not: they are quite sensible about 
how much mental effort—given their limited resources— 
they do invest. Often a first mental model is sufficient to 
understand the information, and very often finding an 
optimal path or time plan takes too much effort. Optimal- 
ity is not necessary in our everyday lives; avoiding mental 
depletion is—it can be dangerous. From a strictly norma- 
tive perspective, humans may not always find the correct 
inference—but they often find one that suffices. 
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13.3 Visualization and Rationality 
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Summary 


The epistemic function of visualization was always con- 
troversial. This chapter is organized around the impor- 
tant, but often blurred, distinction between external and 
internal visualization. External visualizations are visual 
aids such as pictures, graphics, or diagrams, which are 
designed to help people to reason accurately. The ques- 
tion is whether such external visualizations are useful or 
harmful when people solve epistemic problems. Internal 
visualizations are picture-like mental images, a special 
form of mental representation, characterized as analo- 
gous to representations that arise in the mind as a result 
of actual visual perception. The question is whether such 
visual mental images are useful or harmful in rational 
reasoning. The chapter presents the empirical findings 
from cognitive psychology, cognitive brain research, and 
computational modeling. 


1. Two Types of Visualization 


Visualization is an important concept in the cognitive 
sciences. Many experiments show that mental visualiza- 
tion can help remembering objects or events, or when 
objects must be mentally inspected or manipulated 
(Kosslyn, 1980). Such visualizations are correlated with 
activity in early visual cortices, which might explain 
the experienced similarity between mental visualization 
and visual perception (Kosslyn, 1994). However, there 
is also evidence that visualization can disrupt cognitive 
processes and impede thinking. For instance, individuals 
on the autism spectrum often posit that they use visu- 
alization for tasks that typically developing individuals 
perform verbally (Kunda & Goel, 2011). In her famous 
autobiography Thinking in Pictures, Temple Grandin 
(1995/2006) describes how her visual thinking style 
supported her practice but also resulted in problems in 
dealing with more abstract thoughts. Certain psychotic 
drugs that suppress visual mental images (e.g., benzo- 
diazepine) lead to better reasoning performance if the 


problems would otherwise elicit visual images (Pompéia, 
Manzano, Pradella-Hallinan, & Bueno, 2007). And the 
role of visualization in mathematics is still controver- 
sial (Arana, 2016; Mancosu, 2005). Some scholars assert 
that it helps. For instance, the influential mathemati- 
cian Felix Klein (1979) claimed that “mathematics is not 
merely a matter of understanding but quite essentially a 
matter of imagination” (p. 207). Other eminent math- 
ematicians argued that visualization is a nuisance in 
getting mathematical insight or proofs. The most promi- 
nent advocate of this view was Hilbert, who stated that 
deduction should be independent of figures in order to 
be rigorous (Hilbert, 1899). 

This chapter focuses on the relationship between epis- 
temic rationality and visualization. The first concept, 
epistemic rationality, is concerned with belief acquisition, 
formation, and revision. A central component of these 
processes is reasoning, a cognitive process that leads from 
given premises to a rationally justified conclusion, that 
is, anew belief or, at least, a belief that is made explicit in 
the process of inference. To be epistemically rational, the 
conclusion should conform to certain normative ideals, 
which often come from logic or other formal systems for 
rational reasoning. In chapter 5.4 by Gazzo Castaneda 
and Knauff (this handbook), it is shown that classical 
logic might not always be the best normative standard 
for everyday human reasoning. Yet, understanding how 
humans reason logically is still important for a compre- 
hensive cognitive theory of human rationality. 

The second concept, visualization, is often used in a 
very broad sense. Here I use it with two specific mean- 
ings. External visualizations are visual aids such as pic- 
tures, graphics, or diagrams, which are designed to help 
people to reason accurately. The question is whether 
such external visualizations are useful or harmful when 
people have problems to solve or make inferences. 
Internal visualizations are picture-like mental images, a 
special form of mental representation, characterized as 
analogous to representations that arise in the mind as a 
result of visual perception. The question is whether such 
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visual mental images are epistemically useful or harm- 
ful in human logical reasoning. Does visualization help 
people in drawing rationally justified inferences, in rea- 
soning accurately and without logical errors? In other 
words, what are the epistemic benefits and limitations 
of mental imagery in human reasoning? To answers 
these questions, the chapter reports empirical evidence 
from cognitive psychology, cognitive brain research, 
and computational modeling. It closes with some gen- 
eral corollaries on the epistemic role of visual thinking 
in human rationality. 


2. Epistemology of Pictures, Graphics, and Diagrams 


Philosophers, logicians, and mathematicians have always 
had an ambiguous relationship with external visualiza- 
tion, that is, graphics or diagrams (Kramer, 2016). Take, 
for example, the following problem: imagine a square 
with another square inside it, where the corners of the 
inner square are exactly at the midpoints of the sides of 
the outer square. What is the size of the inner square in 
comparison to the outer square? It will be hard for you to 
solve this problem in your head alone. Now the picture 
in figure 13.3.1 is presented to you. You will notice that 
you can now solve the problem more easily—the size of 
the outer square is exactly twice as large as that of the 
inner one, because the four triangles in the corners of the 
outer square together cover exactly the inner square. You 
can easily see this if you imagine folding over the corner 
triangles along the sides of the inner square. 

Pedagogues and teachers often use such visualiza- 
tions in their classes, because they believe that they help 
students to understand complex mathematical issues 
(Kadunz & Yerushalmy, 2015). Yet, in geometry, diagrams 
often have been distrusted and considered inadequate 
and inappropriate for constructing proofs (Giaquinto, 
2016). Imagine, for instance, that I have been just a bit 


Figure 13.3.1 
What is the size of the inner square in relation to the outer square? 
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sloppy with the drawing—then it could be that the four 
triangles do not exactly fit into the inner square, and you 
may not understand the general mathematical princi- 
ple. Apparently, the formal laws of geometry allow you 
to say something about all squares and rectangular tri- 
angles, independently from the concrete visualization— 
your insights are a priori and hence independent from 
perceptual experience, as Kant would say. The long and 
interesting history of the controversies about the episte- 
mology of visualization in mathematics is described in 
Arana (2016), Giaquinto (2016), and Mancosu (2005). 

Since logical reasoning is a particularly important 
foundation of epistemic rationality, it is not surprising 
that philosophers and logicians have paid particular 
attention to the relation between logic and visualization. 
On the one hand, up to the turn of the 20th century, 
mathematicians and logicians were concerned with the 
formal and normative dimension of visualization in 
logical reasoning. Euler and Venn developed logical sys- 
tems for reasoning with diagrams. Most notably, Peirce 
introduced the notion of iconicity in his nonverbal logic. 
An icon, in Peirce’s semiotics, is a sign that is perceptibly 
similar to the object it represents. More recently, Shin 
(1994) has devised a formal syntax and semantics of par- 
ticular forms of Venn diagrams that has also inspired 
much research in artificial intelligence. A good overview 
of this work is given in chapter 13.1 by Jamnik (this 
handbook). 

On the other hand, the majority of scientists and 
philosophers questioned the importance of visualiza- 
tions for logical reasoning. Driven by Ryle (1949) and 
notably the later Wittgenstein (1953/2001), this for- 
mal movement resulted in picture-free logical systems 
that are inextricably linked to language. Since then, 
the dogma for most logicians and philosophers has 
been that rational reasoning relies on the sentences 
of a language from which further valid sentences of 
this language are derived. The language need not be a 
natural language but rather can be any language that 
has a formal grammar, such as propositional or predi- 
cate logic. Thus, external visualizations (e.g., diagrams) 
are often accepted as aids to foster logical intuitions 
but not as an independent tool for conducting logical 
proofs. 

The empirical evidence from cognitive research is 
similarly ambiguous. On the one hand, some experi- 
ments show that diagrams can help in logical thinking. 
Bauer and Johnson-Laird (1993) presented their partici- 
pants conditional reasoning problems as sentences and 
additionally clarified the logical connections using dif- 
ferent types of diagrams. The diagrams were relatively 
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abstract line-drawings that represented the logical rela- 
tions between different sets of entities in a spatial way. In 
another experimental condition, there were no diagrams; 
the tasks were presented only as sentences. The partici- 
pants solved the tasks better with diagrams than with- 
out. On the other hand, Stenning (2002) has explored 
in detail the logical role of visual presentation formats in 
Hyperproof, a tutorial system developed by the logicians 
Barwise and Etchemendy (1994) to teach students formal 
logic through language and visualization. 

Stenning (2002) found that students with poorer 
logical abilities use visualizations, whereas students with 
better logic skills use more nonvisual reasoning strate- 
gies. In fact, strong “visualizers” could benefit the most 
from learning to use abstract strategies instead of visual- 
ization. Similar results were reported by Sato, Sugimoto, 
and Ueda (2017), who explored the usefulness of dif- 
ferent grades of concreteness in conditional reasoning. 
They either presented photos of real objects to their par- 
ticipants, or the objects were presented more abstractly 
in an annotated virtual environment. Reasoning perfor- 
mance was better with the more abstract objects in the 
annotated virtual environment. The visual details of the 
photos seem to distract people from what is actually rel- 
evant for performing the inference. Similar results were 
found in spatial cognition research, where people have 
to make inferences to understand verbal route descrip- 
tions or navigate through photorealistic virtual environ- 
ments (Meilinger, Knauff, & Bilthoff, 2008). 

Overall, the cognitive advantages and disadvantages 
of external visual aids seem to depend on the iconicity, 
concreteness, and abstractness of the form of represen- 
tation. Too many visual details seem to hinder the pro- 
cess of reasoning, whereas more abstract diagrams that 
aid in understanding the underlying logical structure 
of the problem can help people to reason more accu- 
rately and to grasp the intuition of the proof, that is, 
to understand why the theorem and even the proof are 
correct (see chapter 13.1 by Jamnik, this handbook). In 
any case, the common intuition that it is always help- 
ful for understanding and reasoning to make abstract 
issues more concrete in pictures or diagrams is certainly 
wrong. Images, photos, and other highly visual presen- 
tation formats can actually lead to the deterioration 
of epistemic rationality. A possible explanation is that 
people learn better when pictures are excluded from a 
learning system rather than included. The reason is that 
visual overload may result in redundancy, for instance, 
if texts and images are presented simultaneously, which 
can hinder learning and reasoning effectiveness (Mayer, 
2009; Schnotz & Bannert, 2003). This also agrees with 
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the seductive details effect: textbooks are often enriched 
with nice-looking pictures or other eye-catching details 
that are hardly relevant for meaningful learning. But 
empirical findings show that students display poorer 
learning outcomes when they learned with this kind 
of seductive information (Eitel, Bender, & Renkl, 2019; 
Harp & Mayer, 1998). 


3. Epistemology of Visual Mental Imagery 


We have seen that the epistemic role of external visual- 
ization for human reasoning is complex. But how help- 
ful are internal visualizations for logical reasoning? This 
is even more difficult, as we will now see. The idea of 
thinking in the mind’s eye already existed long before 
it received attention in psychology. In philosophy, the 
concept already appears in the oldest writings about 
cognition—the works of Plato and Aristotle (Nigel, 2018). 
Later, philosophers like Descartes, Locke, and Hume also 
developed different thoughts about the structure and 
function of mental imagery. Although their conceptions 
were very different, all had the idea that human thought 
relies on pictorial mental images (Thomas, 1997/2019; 
Tye, 1991). This idea was also supported by the introspec- 
tive reports of scientists, for example, the distinguished 
Austrian physicist Ludwig Boltzmann, who wrote, “I 
am of the opinion that the task of theory consists in 
constructing an image of the external world that exists 
purely internally and must be our guiding star in thought 
and experiment” (Boltzmann, 1890, p. 33). Similarly, 
the chemist August Kekulé reported that visualization 
had suggested to him that the benzene molecule might 
have a cyclic structure (Kekulé, 1865). Today, some con- 
temporary scholars even allege that vivid imagination 
was crucial in solving the key problems of chemistry in 
the 19th century (Rocke, 2010). 

In psychology, several academic debates revolved 
around the role of visual images in human cognition. 
On the one hand, in 1910, studies by Cheves Perky 
revealed that people often merge mental images and 
what is actually seen. Obviously, visualizations can be so 
similar to real perceptions that they can be mistaken for 
the latter (Perky, 1910). On the other hand, in particular 
the “Wurzburg School” argued that thinking is possible 
without imagination. These researchers conducted sev- 
eral experiments in which they asked the participants 
what had happened in their mind when solving different 
kinds of reasoning problems. Not one of them reported 
experiencing visual images. Karl Biihler, one of the main 
figures of this school, concluded that thinking is possi- 
ble without seeing in the mind’s eye. Human thinking, 
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he argued, often is “pure thinking,” which is indepen- 
dent from concrete imagination or visualization (Bihler, 
1909). Other authors have questioned this approach and 
had no doubt that thinking calls for visual imagination 
(e.g., Titchener, 1909). 

Later, psychologists avoided the concept of visual 
mental imagery, given the hostile criticism it had received 
from behaviorists (Watson, 1913). In his famous “mani- 
festo,” Watson (1913) argued that the controversy about 
visual mental images was a prime example of the maladies 
of psychology for which behaviorism would be the ther- 
apy. For Watson, the speculations about mental images 
in the mind (as all other mentalistic concepts) were only 
“old wives’ tales” and nothing more than the sentimental 
“dramatizing” of verbally mediated memories (Watson, 
1928). 

This radically changed with the rise of cognitive psy- 
chology, which put mental representations and pro- 
cesses center stage. One of the most striking arguments 
for the role of visual mental images in human cognition 
was the seminal experiment by Kosslyn, Ball, and Rei- 
ser (1978), who asked people to memorize and visualize 
the map of a fictitious island. In a later memory test, 
the time to mentally move from one object to another 
was proportional to the distance between the objects on 
the map. Such results were taken as evidence that visual 
mental images are a distinct kind of mental representa- 
tion, which is similar to representations resulting from 
visual perception (e.g., Pearson, Naselaris, Holmes, & 
Kosslyn, 2015). 

Other researchers are, on the whole, dismissive of 
the role of visualization in human thought. Largely in 
parallel to the arguments of logicians against external 
diagrams for logical reasoning, these scholars argue that 
mental reasoning is solely based on propositional repre- 
sentations as a language of thought. Visual imagery is a 
mere epiphenomenon that does not play a causal role 
in mental processes (e.g., Adler & Rips, 2008; Pylyshyn, 
2002; Rips, 1994). Under the label “mental imagery 
debate,” these controversies addressed some of the most 
fundamental questions at the frontiers of the cognitive 
sciences. They are still not resolved. In the following, 
the empirical results from the area of logical reasoning 
are reported. 


3.1 Results from Cognitive Psychology 

A pioneering study on the role of visual images in logi- 
cal reasoning was carried out by De Soto, London, and 
Handel (1965), who argued that reasoners visualize the 
content of reasoning problems in a mental image and 
then “read off” the conclusion by inspecting this visual 
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image. Following this idea, several authors assumed that 
if reasoning relies on visual mental images, then reason- 
ing with problems that are easy to visualize should be 
easier and accompanied by better performance than rea- 
soning with problems that are hard to visualize. Yet, until 
recently, the evidence was equivocal. Some researchers 
found that the ease of visualization helps people to rea- 
son accurately, whereas others did not find such an effect. 
A detailed overview of these ambiguous findings is given 
in Knauff (2013). 

Research from our own laboratory helps to resolve the 
inconsistency in the previous results and draws an oppo- 
site picture to what the mental imagery account suggests: 
visual mental images can actually impede reasoning and 
hinder the process of inference. If the content of a rea- 
soning process is easy to imagine visually, people need 
more time and make more errors than with less visual 
problems (Knauff, 2013). In Knauff and Johnson-Laird 
(2002), we empirically identified four sorts of reason- 
ing problems: (1) visuospatial problems, which are easy 
to envisage visually and spatially; (2) visual problems, 
which are easy to envisage visually but hard to envisage 
spatially; (3) spatial problems, which are hard to envisage 
visually but easy to envisage spatially; and (4) abstract 
problems, which are hard to envisage either visually or 
spatially. We then asked our participants to solve rea- 
soning problems of these four sorts and measured the 
error rates and how much time they needed to solve the 
problem. A theory based on visual mental images would 
predict that visual (and probably visuospatial) problems 
are easier to solve. Our prediction, however, was that 
problems that elicit visual images containing details that 
are irrelevant to an inference should impede the process 
of reasoning. Our findings supported this prediction: in 
several experiments, we found that the ease of visualiza- 
tion impaired reasoning: participants were significantly 
slower with these problems than with the other sorts 
of problems. In other experiments, we also found that 
visualization results in lower reasoning accuracy: people 
make more errors with visual problems than with the 
other sorts of problems (Knauff, 2009; Knauff & May, 
2006). This is called the visual impedance effect (Knauff, 
2013; Knauff & Johnson-Laird, 2002). 

The visual impedance effect was corroborated in 
research from several other groups. For instance, stud- 
ies have shown that one of the main causes for dys- 
lexia (a reading deficit) is the strong tendency of 
dyslexics to represent the information from a text in a 
visual, rather than a verbal, way. In a series of experi- 
ments, Bacon and coworkers could show that dyslexics 
are also handicapped in logical reasoning with highly 
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visual materials. The authors explain this with the 
visual impedance effect (Bacon & Handley, 2010; Bacon, 
Handley, & McDonald, 2007). Panagiotidou, Serrano, 
and Moreno-Rios (2018) asked adult participants with 
reading deficits to solve reasoning problems, which were 
represented either visually or verbally. They also found a 
strong visual impedance effect and argue that the visual 
materials help to study the reasoning abilities of people 
with and without reading deficits. Tse, Ragni, and Losch 
(2017) found that the forming of excessive visual images 
induced by the premises can impede reasoning and that 
this interacts with the number of relations used in the 
problem. Gazzo Castafieda and Knauff (2013) used the 
well-known Verbalizer—Visualizer Questionnaire (VVQ; 
Richardson, 1977) to identify two groups of individu- 
als with different cognitive styles. One group consisted 
of “visualizers,” with a strong tendency toward report- 
ing visualization during thinking, and the second group 
consisted of “verbalizers,” with a strong bias toward 
reporting verbalization during thinking. We found that 
visualizers showed a strong visual impedance effect with 
the visual problems but not with nonvisual problems. 
This effect was stronger than in the nonvisualizers. In 
Mathias, Vogel, and Knauff (2019), we found similar 
effects of visual thinking styles in the learning of mini- 
mally invasive surgery. 


3.2 Results from Cognitive Neuroscience 

Several studies showed that areas of the primary and 
secondary visual cortex are active during visual mental 
imagery. This supports the assumption that visual per- 
ception and visual imagination also overlap on the cor- 
tical level (Kosslyn, Ganis, & Thompson; 2001; Kosslyn 
& Thompson, 2003). Other studies explored the neural 
correlates of human reasoning. A core result is that rea- 
soning problems activate a complex and widespread 
network of cortical areas, covering regions in the fron- 
tal, temporal, and parieto-occipital lobes (Goel, 2007; 
Knauff, 2009; Prado, Chadha, & Booth, 2011). 

Our group was the first that studied the connection 
between reasoning, visual mental imagery, and the visual 
cortices. In one functional magnetic resonance imaging 
(fMRI) study, we used the same materials as Knauff and 
Johnson-Laird (2002). While lying in the scanner, par- 
ticipants saw the visual, the visuospatial, the spatial, and 
the abstract problems and had to evaluate conclusions 
by mouse clicks (Knauff, Fangmeier, Ruff, & Johnson- 
Laird, 2003). Two results were important: first, during 
the solving of all problems, we found activity in areas 
of the parietal cortex. This indicates that people solved 
all kinds of problems by constructing and inspecting 
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spatially organized mental representations, which are 
more abstract than visual images (Knauff, 2013). How- 
ever, the second result was that only the visual prob- 
lems resulted in additional activity in the early visual 
cortex. These areas are related to visual mental imagery 
in the literal sense, because they are the only regions 
of the brain that are retinotopically organized, that is, 
a stimulation of the retina is (almost) analogically rep- 
resented in these brain areas. Obviously, these areas can 
be activated not only by visual perception but also by 
the process of thought. Our interpretation is that, first, 
the visual problems activate visual mental images in 
these visual areas but, second, as we found before, these 
mental images can impede the process of reasoning. 

In another study, we used transcranial magnetic 
stimulation (TMS; Hamburger et al., 2018). TMS uses 
strong magnetic fields to hinder the neural processing in 
a well-defined cerebral area and to study how this sup- 
pression influences participants’ performance in cogni- 
tive tasks. In the experiment, we applied the magnetic 
field to the primary visual cortex. Our assumption was 
that this should hinder participants’ ability to create 
visual mental images in the visual cortex, and this in 
turn should help them to reason better than with the 
visual images they would spontaneously construct, if 
the activity in these areas were not suppressed. That is 
exactly what we found. When people could not visual- 
ize the content of the reason problem, they reasoned 
better—which is exactly the opposite of what the visual- 
ization theory would assume. In another TMS study, we 
applied the TMS signal to the parietal cortex and found 
that this hinders people’s reasoning performance. They 
reasoned less accurately when they could not construct 
spatial mental models in the parietal cortex (Ragni, Franz- 
meier, Maier, & Knauff, 2016). 

We also used patient studies to explore the visual 
impedance effect. In one of the studies, we asked con- 
genitally totally blind people to solve our reasoning 
problems. People who are blind from birth do not experi- 
ence visual mental images. However, they are as good as 
sighted people in constructing and utilizing spatial repre- 
sentations. Again, the participants solved visual, visuo- 
spatial, spatial, and abstract problems. Interestingly, the 
visual impedance effect disappeared in those participants. 
Obviously, congenitally totally blind people are immune 
to the visual impedance effect, as they do not construct 
impeding visual mental images (Knauff & May, 2006). 

To more precisely study the relation between visual- 
ization and reasoning processes, we conducted a further 
fMRI study (Fangmeier & Knauff, 2009; Fangmeier, Knauff, 
Ruff, & Sloutsky, 2006). This was the first experiment 
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that was conducted in a way that allowed us to distin- 
guish between different phases of a reasoning process. 
We distinguished between three different phases: first, 
people have to understand the premises. Second, they 
have to integrate the information from the premises 
into one unified mental representation. Third, they are 
confronted with a conclusion and have to say whether 
or not this conclusion logically follows from the given 
premises. As shown in figure 13.3.2, we obtained differ- 
ent patterns of neural activity for all three phases. In the 
first phase, we found activity in the primary visual cor- 
tex, although all activities related to the purely visual 
presentation of premises were excluded. In the second 
phase, this activity was still visible, together with some 
other activities that were related to executive control, 
and so forth. In the third phase, which is the actual 
core of reasoning, something interesting happened: 
now the activity in the visual cortex disappeared, but 
another activity in a large cluster in the parietal cortex 
arose. We argued that this pattern of neural activation 
explains why people experience visual mental images 
during reasoning. During the reading and processing 
of the premises, such visual mental images are present 
in the visual cortex. This might then help people to 
maintain the information active in working memory. 
Hence, people have the feeling that they can see the 
reasoning problem in their inner eye. However, during 
the actual reasoning, this activity was not present any- 
more. Instead, cortical areas were active that are related 
to spatial processing from different modalities. Typically, 
people do not become aware of such spatial represen- 
tations, because they do not have conscious access to 
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Figure 13.3.2 
Neural activation in the three phases of reasoning (from Fang- 
meier et al., 2006). 
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representations and processes in the parietal cortex (for 
details, see Knauff, 2013). 


3.3. Results from Computational Cognitive Modeling 
The idea that reasoning is linked to visualization also 
aroused researchers’ interest in developing computer 
programs of imagery-based inference. A classic example 
is the WHISPER system, which could observe and manip- 
ulate diagrams to solve problems in the blocks-world 
environment (Funt, 1980). Kosslyn and Shwartz (1977) 
programmed a system in which the locations of objects 
were represented metrically in a matrix of filled and 
unfilled cells, simulating the use of visual images. Today, 
many different imagery-based approaches in artificial 
intelligence exist, notably in the area of diagrammatic 
reasoning, where researchers seek to use visual images 
to solve complex problems. A good overview is given in 
chapter 13.1 by Jamnik (this handbook). 

Some computer scientists have developed systems for 
logical reasoning with visual mental images and with 
more abstract spatial representations. These computer 
simulations are helpful in exploring whether visual 
mental images or spatial representations are more useful 
on the algorithmic level of human reasoning. One such 
system was developed by Berendt (1996), who tried to 
simulate results from our research group. In these exper- 
iments, we showed that in the case of ambiguous sets 
of premises, people prefer one model but ignore other 
interpretations. The preferred model is the one that is 
easier to construct and manipulate in working memory 
(Knauff, 1999; Knauff & Ragni, 2011; Knauff, Rauh, & 
Schlieder, 1995; Rauh et al., 2005). Berendt could show 
that a system that uses visual mental images is able to 
reconstruct some of these preferences (Berendt, 1996). 
However, Schlieder (1999) developed another com- 
putational model that just used the order of starting 
points and endpoints of the object’s edges. This means 
that this representation is spatial in nature—it just 
needs ordering information—and it is more parsimoni- 
ous than Berendt’s visual system, which uses excessive 
visual details to perform the inference. Nevertheless, 
the abstract spatial model was better able to reconstruct 
the empirical data than the visual model (Schlieder & 
Berendt, 1998). 

In Ragni and Knauff (2013), we developed a compu- 
tational theory and implemented computer program for 
reasoning with spatial mental models. The prisM system 
can reconstruct many empirical findings from the litera- 
ture on human reasoning. The basic idea is that working 
memory is conceptualized as a spatial array of filled and 
unfilled cells. A spatial scanning mechanism inspects this 
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array to find new information, which is not explicitly 
given in the premises. In this way, the system can draw 
logically valid conclusions without representing visual 
details. It can also reconstruct why people sometimes err 
and draw inferences that are invalid from a logical point 
of view. Based on this work, two approaches tried to sim- 
ulate the visual impedance effect in computational sys- 
tems (Albrecht, Schultheis, & Fu, 2015; Boeddinghaus, 
Ragni, Knauff, & Nebel, 2006). The approaches are an 
interesting starting point, although still much work has 
to be done to understand the algorithmic details of the 
visual impedance effect. 


4. Corollaries and Future Work 


The goal of this chapter was to develop a better empiri- 
cal understanding of the epistemic role of visualization 
in human rationality. This is important, as the field is 
full of speculations from folk psychology and mislead- 
ing subjective experiences. Sometimes these may even 
bias scientists toward certain theories. So, what can we 
really learn from this chapter? 

The first corollary is that introspection can be fatally 
misguiding. This is by no means new. It has long been 
known that people do not have conscious access to most 
of their cognitive processes. 

The second corollary is that the visual impedance 
effect provides a new view on the ongoing debates on the 
epistemic role of visualization in logical reasoning and 
rationality. Most of these controversies have taken place 
in philosophy, mathematics, and other areas addressing 
the normative function of visualization. Instead, the pre- 
sent chapter is concerned with actual human reasoning. 
As such, it is part of the goal to develop a comprehensive 
descriptive theory of human thinking. Such a theory also 
accounts for errors, that is, deviations from what we nor- 
matively would call rational. 

The third corollary is that visual mental imagery in 
reasoning is not a mere epiphenomenon: it has a causal 
power. But this effect is not in the direction that the 
orthodox idea of visual imagery suggests. Visualization 
does not help people to reason; actually, it can hin- 
der human reasoning. The reason is that, in the “fore- 
ground” of conscious experience, people seem indeed to 
envisage the content of a reasoning problem in a visual 
image, at least if the situation is easily visualizable. Since 
the visual cortex is linked to visual experience, these 
mental images might be pushed into consciousness. 
However, in the “background” of conscious experience, 
the actual logical work is based on more abstract mental 
representations and processes. Hence, premises that 
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almost automatically evoke vivid visual images initiate a 
“detour” via the visual cortex, which requires additional 
time but is useless for the actual reasoning process that 
works on these abstract representations. This detour may 
cause the visual impedance effect and may also cause the 
low utility of external visualizations in the form of pic- 
tures or other highly visual illustrations (Knauff, 2013). 
This largely resembles the normative view of most logi- 
cians and philosophers that reasoning should be inde- 
pendent of visualization in order to be rational. 

The fourth corollary concerns the impact of the cur- 
rent results for education in logical reasoning. This 
chapter does not underestimate the power of mental 
visualization in training and education. We know from 
many different areas that mental imagery can acceler- 
ate learning and leads to better performance in different 
tasks (Nigel, 2018). The point in this chapter is, however, 
that the power of visualization depends on many dif- 
ferent factors. In particular, very concrete, vivid mental 
imagery must be distinguished from more abstract spa- 
tial representations. Although many people are confi- 
dent that the former is important for human cognition, 
it might actually be the case that the facilitating effects of 
imagination are mostly caused by more abstract—albeit 
not consciously experienced—spatial representations 
and processes. 

The fifth corollary is that the relation between exter- 
nal and internal visualizations is complex. Sometimes 
the former can trigger the latter. And this means that it 
is important how much visual detail is given in external 
visualizations such as diagrams, figures, pictures, and so 
forth. Too many seductive visual details in external visu- 
alizations may trigger the construction of visual mental 
images, which can impede cognition. But if external 
visualizations really focus on the information relevant 
for the task, they can support thinking and learning (see 
chapter 13.1 by Jamnik, this handbook). Teachers and 
pedagogues should carefully consider this important 
trade-off between eye-catching, nice-looking pictures 
and meaningful learning. 

The sixth corollary from this chapter is that still 
much work has to be done in the field. On the one hand, 
logical reasoning is a very particular domain of human 
cognition. Hence, we probably cannot simply transfer 
the corollaries from this chapter to all kinds of problems. 
On the other hand, however, the reported results chal- 
lenge the positive image that visualization has in science 
and folk psychology. I hope that this result will trigger 
further research that goes beyond the area of logical rea- 
soning and human rationality. 
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Section 14 Scientific Rationality 


14.1 Scientific Rationality and Objectivity 


Line Edslev Andersen and Hanne Andersen 


Summary 


This chapter provides an overview of the accounts of sci- 
entific rationality and objectivity that have been offered 
by philosophers of science during the 20th century. We 
begin by presenting different accounts of how individ- 
ual scientists should act to be rational and objective. In 
the latter half of the 20th century, some philosophers 
argued that the rationality and objectivity of science are 
obtained at the level of communities of scientists rather 
than at the level of individual scientists. Hence, we also 
present accounts of how communities of scientists can 
be rational and objective. Finally, we illustrate how the 
philosophical discussions about scientific objectivity 
may be relevant for scientific practice. 


1. Models of Individual Rationality and Objectivity 


This chapter provides a brief overview of the various 
accounts of scientific rationality and objectivity that 
have been offered by philosophers of science during the 
20th century. We start with models of individual ratio- 
nality and objectivity, which dominated philosophy of 
science until the mid-20th century. We then move to 
models of group rationality, which became popular dur- 
ing the second half of the 20th century. We also examine 
recent models of group objectivity, focusing on feminist 
approaches to objectivity in science. Finally, we illustrate 
how the philosophical discussions about objectivity 
may be relevant for scientific practice. In line with most 
English-language accounts of rationality and objectiv- 
ity in science, “science” is used to refer to the natural 
sciences. 

Science is often considered the epitome of ratio- 
nality and objectivity. The view that science is objec- 
tive can be understood in two different ways. It can 
be understood either in the way that scientific theories 
faithfully describe facts about the world or in the way 
that the methods employed in science are independent 
of social and personal values and biases (see Reiss & 


Sprenger, 2014/2017). In this section, we focus on mod- 
els of rationality and objectivity from the first half of 
the 20th century, and at that time, science was gener- 
ally seen as objective in both these ways. The methods 
employed by scientists were usually seen to be objective 
and to lead to objective theories. In fact, insofar as sci- 
entists acted rationally, they were usually considered to 
adhere to objective methods that led to objective theo- 
ries. Thus, the central question was what it means for 
scientists to act rationally. 

In the first half of the 20th century, many philosophers 
of science linked questions about scientific rationality 
to the question of how individual scientists validate 
their scientific beliefs. This approach was based on the 
assumption that there is a clear distinction between dis- 
covery and justification in science.' On this view, how 
scientists form new scientific hypotheses is seen as an 
empirical question to be relegated to disciplines such as 
psychology, sociology, and history of science, while in 
contrast, the question of how scientists should evaluate 
scientific hypotheses and make choices between com- 
peting theories is seen as a normative question, to be 
answered by philosophy. The standard view was that sci- 
ence could be characterized by its special method that 
specified how scientific theories could be assessed. On 
this view, scientists act rationally by adhering to this sci- 
entific method. Consequently, an important philosophi- 
cal question was how to define this method. 

Some of the dominant accounts of the unique scientific 
method in the 20th century have been Hempel’s (1966) 
hypothetico-deductive model, Popper’s (1963/1972, 
1934/2014) falsificationist model, and Bayesianism (we 
will not examine Bayesianism in the present chapter but 
refer the reader to chapter 4.1 by Hajek & Staffel, this 
handbook). 

Hempel’s hypothetico-deductive, or H-D, model pre- 
sents a logic of confirmation. On this model, scientists 
assess a hypothesis by first deducing from it the observ- 
able events that would take place if the hypothesis 
were true and then conducting experiments to observe 
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whether the predicted events occur. If the predicted 
events occur, this provides supporting evidence for the 
hypothesis, where the degree of support depends on the 
quantity, variety, and precision of the supporting evi- 
dence. In contrast, if the predicted events do not occur, 
scientists should reject the hypothesis in question. 

In contrast to the H-D model’s logic of confirma- 
tion, Popper’s falsificationist model presents a logic of 
refutation. On the falsificationist model, like on the H-D 
model, scientists start by deducing from the hypothesis 
under test the observable events that would occur if the 
hypothesis were true. However, Popper stressed that, 
regardless of the amount of supporting evidence for 
the hypothesis, the hypothesis can never be confirmed 
with certainty. By contrast, it follows logically that the 
hypothesis is false if it has false implications: if events 
can be deduced from it that do not in fact occur. This 
led Popper to suggest that, for scientists to act rationally, 
they must proceed by trying to falsify hypotheses. Thus, 
Popper focused on the potential falsifiers of a hypothesis 
and applauded scientists’ sincere attempts at falsifica- 
tion. On this view, the more a hypothesis claims, the 
better it is, in the sense that it is more open to falsifi- 
cation. If a hypothesis has been subjected to numerous 
tests and still has not been falsified, this does not jus- 
tify scientists in accepting the hypothesis. It only means 
that the hypothesis should be refined to be more open 
to falsification and be made subject to even more strin- 
gent tests. On this view, the scientific method can be 
described as a method of conjectures and refutations. 

Both the H-D model and falsificationism have had 
a huge influence on the perception of science and sci- 
entific rationality. In general, the view that there is a 
unique scientific method, to which all rationally behav- 
ing scientists adhere, has been strong not only in phi- 
losophy but also in other fields. For example, in science 
education, the scientific method is often presented as a 
fixed procedure from the description of a phenomenon, 
over the formulation of a hypothesis that explains the 
phenomenon, to the conduct of an experiment that 
tests the hypothesis (Blachowicz, 2009; see also Bauer, 
1992, for a historical overview).’ The literature on sci- 
entific method is vast, but overviews can be found, for 
example, in H. Andersen and Hepburn (2015/2016), Lau- 
dan (1968), and Losee (2004, 2005). 

We end this section by discussing some challenges to 
the presented models of individual rationality. 

First, it has been argued that the rational acceptance 
or rejection of a hypothesis is not solely based on experi- 
ments but is to be seen as a value-based decision between 
different courses of action, where the values may be 
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epistemic or nonepistemic, namely, social, ethical, and 
political values. An aspect of this debate is therefore also 
whether to understand rationality in epistemic terms as a 
question of which theory or hypothesis to accept, given 
the evidence, or if rationality can also be understood 
in instrumental terms as a question of which actions to 
perform, given specific beliefs and ends. Among philoso- 
phers of science, Churchman (1948) and Rudner (1953) 
have argued that an analysis of the methods of scientific 
inference includes both an assessment of the strength of 
the evidence in favor of the hypothesis and an assess- 
ment of the consequences of making a mistake in accept- 
ing the hypothesis. A similar debate has been seen within 
statistics. Here, Fisher (1955) has argued that a theory of 
inductive inference should provide a numerical expres- 
sion of disbelief in a hypothesis, while Neyman (1956) 
and Pearson (1955) have argued that the consequence of 
error also needs to be included (for details on this debate, 
see, e.g., Howie, 2002; Lenhard, 2006; Marks, 1997). In 
recent decades, these issues have been pursued by dis- 
cussing whether there is a distinction between rational 
and irrational influences on theory choice and, more 
specifically, between the roles played by epistemic and 
nonepistemic values (see, e.g., Douglas, 2009; Machamer 
& Wolters, 2004). For further reading on this topic, the 
reader is referred to chapter 14.2 by Bueter (this hand- 
book) on the value-freedom of science. 

Second, it has been argued that there is no single ratio- 
nal method for scientists to follow, because scientific 
norms and aims differ over time and across disciplines. 
Hence, it has been argued, it is not possible to provide a 
universal and ahistorical account of the rational method 
of scientific inquiry. During the 1960s and 1970s, his- 
torically inclined philosophers of science such as Fey- 
erabend, Hanson, Kuhn, Laudan, Shapere, and Toulmin 
argued from studies of the history of science that sci- 
entific norms and aims may differ across disciplines or 
over time.* The most radical position was advanced by 
Feyerabend, who in a monograph with the provocative 
title Against Method (1975/1993) argued for a method- 
ological anarchism on which any methodological rule 
can be violated fruitfully in some contexts. 

Finally, some, such as Hanson (1958), argued that 
observations are often theory-laden (i.e., they depend on 
theoretical assumptions). The relation between observa- 
tion and theory is therefore more complex than assumed 
both by the accounts of the rationality of science exam- 
ined above and by the view that science is objective in 
the sense that scientific theories faithfully describe facts 
about the world. Like the previous paragraph, this pro- 
vides a reason for believing that it is not possible to 
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provide a universal and ahistorical account of the ratio- 
nality of science. 


2. Models of Group Rationality 


In this section, we consider models of rationality that can 
accommodate the challenges to the traditional models of 
rationality just presented. As opposed to the traditional 
models, these are models of group rationality. In the next 
section, we consider models of group objectivity. During 
the second half of the 20th century, philosophers of sci- 
ence started questioning whether the rationality of 
science should be addressed from the level of the individ- 
ual scientist and not rather from the level of the scientific 
community. 

Especially the work of Thomas S. Kuhn was founda- 
tional for this discussion. Kuhn developed an account 
of the scientific enterprise on which individual scientists 
work on a set of received beliefs that they do not question. 
An individual scientist will sometimes replace this set of 
beliefs with another set of beliefs, and the time at which 
she does so is partially determined by personal prefer- 
ences. Given the traditional conception of scientific ratio- 
nality and objectivity introduced in the previous section, 
this makes science seem irrational and subjective. How- 
ever, on Kuhn’s model, science is a rational and objective 
enterprise, but it is the scientific community that serves 
as the agent responsible for the rationality and objectivity 
of science. 

Kuhn is well known for the phase model of the devel- 
opment of science that he advanced in his monograph 
The Structure of Scientific Revolutions (1962/2012). On this 
model, science develops through successive periods of 
cumulative “normal science” separated by noncumulative 
“revolutions.” Normal science is dependent on some set 
of received beliefs, a “paradigm,” which marks out what 
the acceptable research problems are and what acceptable 
approaches and solutions to these problems must look 
like. Yet, some of the scientific problems defined by a par- 
adigm may turn out recalcitrant. If such anomalies cannot 
be resolved, they may cause a crisis in the scientific com- 
munity, and this crisis may eventually lead to a revolu- 
tion, after which the community will have adopted a new 
paradigm. Such a revolution is not a cumulative process: 
the new paradigm is not just an extension of the old; on 
the contrary, the previous paradigm may have precluded 
solutions of the sort provided by the new. Thus, for Kuhn, 
competing paradigms were incommensurable and could 
not be compared in any straightforward manner. 

This phase model of the development of science 
has often by itself been taken to imply that science is 
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irrational. To understand why it does not, one must look 
at the “essential tension,” as Kuhn called it, between the 
tradition-preserving activity of normal science and the 
tradition-shattering activity of scientific revolutions. On 
the one hand, the tradition-preserving phases of normal 
science are very effective for solving the problems that 
the existing paradigm defines. When scientists within a 
community adhere to the same paradigm, science can 
move faster and penetrate deeper than if they continu- 
ously questioned the paradigm. In this respect, normal 
science increases the efficiency with which scientific 
problems are solved. 

On the other hand, science would not be able to pro- 
duce fundamental innovations if normal science were the 
only mode of doing science. A tradition-shattering com- 
plement is needed as well. Kuhn introduced a mechanism 
that could ensure that the change from one paradigm to 
another happens neither too quickly nor too late. Thus, if 
there is a distribution of conservative and innovative dis- 
positions among the members of a scientific community, 
individual scientists may react differently to encountered 
anomalies, such that only some start considering alter- 
natives to the reigning paradigm while others continue 
working within the established paradigm.* 

To explain both how individual scientists can come 
to divergent decisions and how the scientific commu- 
nity can, in the end, come to a common decision on 
which theory to prefer, Kuhn suggested that theory 
choice was guided by a set of epistemic values that the 
members of the scientific community share, such as 
accuracy, consistency, scope, simplicity, and fruitfulness. 
While these values provide the basis for theory choice, 
they do not determine it: individuals may legitimately 
interpret and weigh the values differently. The fact that 
individuals may weigh the values differently matters 
to theory choice because the values may conflict. For 
example, the simpler of two competing theories is not 
necessarily also the most accurate. Further, Kuhn even 
claimed that not only epistemic values such as accu- 
racy, consistency, scope, simplicity, and fruitfulness may 
play a role in theory choice; also, personal preferences 
about, for example, originality, risk taking, aesthetics, or 
philosophical convictions may influence theory choice 
(see Hoyningen-Huene, 1993, chapter 4.3.c). As a result, 
although individual members of the scientific com- 
munity may employ the same set of shared epistemic 
values, they can nevertheless still reach different conclu- 
sions as to which of the competing theories to pursue. 

Hence, Kuhn’s model of the development of science 
explicitly “requires a decision process which permits ratio- 
nal men to disagree” (Kuhn, 1977, p. 332). This stands 
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in stark contrast to the accounts of philosophers like 
Hempel, who, as described above, linked the rationality 
of science to individual scientists’ adherence to a unique 
method that determines the rational theory choice. On 
this view, two rational scientists will always choose the 
same theory. On Kuhn’s view, the fact that theory choice 
is guided by values that differ from scientist to scientist, as 
well as by shared values that may be interpreted differently 
by different scientists, contributes to the development 
of science. This does not mean that science is irrational; 
rationality just has to be seen from a social rather than 
from an individual perspective. As a new, alternative the- 
ory is developed, it may prove capable of solving encoun- 
tered anomalies in such a way that it is gradually accepted 
by more and more scientists until, eventually, a consensus 
on the new theory is established among members of the 
scientific community. Dissent continues until all mem- 
bers of the community, based on their individual inter- 
pretation and weighing of the shared values, prefer the 
same paradigm. Hence, “it is the community of specialists 
rather than its individual members that makes the effec- 
tive decision” (Kuhn, 1962/2012, p. 199). A new consen- 
sus in the scientific community reflects that arguments in 
favor of the new paradigm have proliferated. 

It is not clear where this leaves us in terms of how 
exactly to conceive of scientific objectivity. Kuhn does 
not go much further than to suggest that an account of 
scientific objectivity should be informed by the fact that 
the criteria of accuracy, consistency, scope, simplicity, 
and fruitfulness constrain, but do not determine, theory 
choice: 


Objectivity ought to be analyzable in terms of criteria like 
accuracy and consistency. If these criteria do not supply all 
the guidance that we have customarily expected of them, 
then it may be the meaning rather than the limits of objectiv- 
ity that my argument shows. (Kuhn, 1977, p. 338, emphasis 
added) 


Hence, Kuhn claims to have shown that science is objec- 
tive in a different sense than traditionally thought, 
rather than having shown that science is subjective. But 
while Kuhn develops an account of scientific rationality 
on which the rationality of science is obtained at the 
level of scientific communities, he at most hints at an 
account on which the objectivity of science is likewise 
obtained at the level of scientific communities. As will 
be described in the next section, others have developed 
such accounts of scientific objectivity. 

When Kuhn introduced his model, many critics found 
that it made theory choice seem irrational (see, e.g., 
Lakatos, 1970; Scheffler, 1967/1982; or Shapere, 1966). 
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This perception was further amplified when sociologists 
of science, partially inspired by Kuhn’s account, started 
arguing that scientists’ beliefs should be explained in 
terms of the social conditions that bring them about. 
This “strong” program of sociology of science claimed 
to be impartial with respect to rationality and irratio- 
nality, meaning that it set out to explain successful as 
well as unsuccessful knowledge claims in science in the 
same type of way, with reference to the same set of social 
causes (Bloor, 1976/1991). 

However, other philosophers of science, such as David 
Hull (1988) and Philip Kitcher (1993), have followed 
Kuhn in arguing that the social structure of a scientific 
community contributes to the effective realization of the 
epistemic goals of science (for a detailed analysis, see also 
Wray, 2011). Hull (1988) argued that scientists are moti- 
vated by the credit they get for contributing true theo- 
ries. If they would get credit for contributing fraudulent 
theories, they would cheat. But the structure of reward 
and punishment in science ensures that scientists will not 
cheat, because the punishment if they are caught is very 
severe. On Hull’s account, science is so successful because 
the social structure of science is such that it is in the self- 
interest of scientists to pursue true theories. 

Kitcher (1993) has similarly argued that nonepis- 
temic pressures on individuals, such as desire for fame, 
can help a scientific community to effectively realize its 
epistemic goals. It may be tempting for all the scientists 
working on a given problem to use the same method to 
pursue the problem: the available method whose intrin- 
sic qualities are highest. But for a scientific community 
to effectively realize its epistemic goals, it is important 
that scientists choose different methods to pursue a prob- 
lem. For it may be an unorthodox method that can pro- 
vide the solution. The nonepistemic pressures can help 
prevent that all scientists working on the problem use 
the same method. For example, if a scientist has a desire 
for fame, it may be better to pick an unorthodox method 
since there is a chance that it will provide the solution, 
and there are fewer direct competitors in the sense of 
other scientists pursuing the same strategy. In this way, 
nonepistemic considerations at the level of individual 
scientists can contribute to the attainment of the epis- 
temic goals of the scientific community. 

On the accounts considered in this section, the agent 
that is responsible for the rationality of science is the 
scientific community as a whole, and consequently, the 
rationality of scientific beliefs should be examined using 
the tools of social epistemology. For the topic of social 
epistemology, the reader is referred to chapter 10.1 by 
Dietrich and Spiekermann (this handbook). 
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Similarly, philosophers of science such as D’Agostino 
(2010), De Langhe (2010, 2012, 2014; see also De Langhe 
& Rubbens, 2015), Weisberg (2013; see also Weisberg & 
Muldoon, 2009), or Zollman (2007, 2010, 2018; see also 
Mayo-Wilson, Zollman, & Danks, 2011) have drawn on 
game theory, economics, and mathematical modeling of 
social behavior in examining how various forms of het- 
erogeneity in the scientific community can be beneficial 
or detrimental for the progress of science. 


3. Models of Group Objectivity 


In recent decades, feminist philosophers of science have 
examined different ways in which traditional models of 
the objectivity of science have disadvantaged women 
and other minority groups (for a review, see Anderson, 
2000/2020). In their view, scientific knowledge has a 
situated and perspectival nature. What is known reflects 
the perspective of the knower; there is no view from 
nowhere. This has led some to embrace subjective ways 
of knowing nature (Irigaray, 1987). Others have argued, 
similarly to Kuhn, that the objectivity of science should 
be assessed at the level of scientific communities (Har- 
away, 1988; Longino, 1989, 1990; Solomon, 2001). 

In Helen Longino’s view, observational data are evi- 
dence for a theory only relative to a context of back- 
ground assumptions. This is so because the theory says 
more than the statements describing the data. The the- 
ory is, in other words, underdetermined by the data (see 
Longino, 1990, pp. 38-61, for her defense of a contex- 
tualist account of epistemic justification). On Longino’s 
account, the background assumptions are often biased.* 
So she raises the question of how scientific inquiry can 
be objective if the background assumptions of indi- 
vidual scientists are biased. She argues that the produc- 
tion of scientific knowledge can be said to be objective 
only if scientific results are subjected to evaluation by 
a plurality of individuals and formed by the ensuing 
discussions. Only then can we see how the influence of 
subjective assumptions and preferences can be blocked 
or mitigated (Longino, 1989, p. 266).° 

Like Kuhn, Longino believes that scientific inquiry as 
practiced is generally objective. Longino has pointed out 
that scientific knowledge is not produced by just collect- 
ing the products of individuals but through a process in 
which those products are critically examined and modi- 
fied by other scientists. Experiments thus “get repeated 
with variations by individuals other than their origina- 
tors, hypotheses and theories are critically examined, 
restated, and reformulated before becoming an accepted 
part of the scientific canon” (Longino, 1989, p. 265). 
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Hence, scientific inquiry is objective because what is ulti- 
mately accepted as a piece of knowledge is the result of 
a process formed by multiple individuals with different 
perspectives. 

At the same time, Longino makes the case that sci- 
entific communities should strive for a higher degree of 
objectivity. Objectivity comes in degrees, and the more 
perspectives represented in the critical discussions that 
form scientific results, the more objective the process. 
This has important implications for the organization of 
science. The degree to which scientific inquiry is objec- 
tive depends on the extent to which the scientific com- 
munity is organized in such a way that a multiplicity 
of perspectives on a theory is elicited. Longino tenta- 
tively suggests a list of criteria that a scientific commu- 
nity must meet in order to support effective criticism. 
Included on the list are appropriate venues for criticism 
and that the community responds to criticism (Longino, 
2006, pp. 172-173). 

Other scholars have argued for a conception of objec- 
tivity comparable to Longino’s (e.g., Haraway, 1988; 
Harding, 1992; Solomon, 1992, 2001; Wylie, 2013). For 
example, Donna Haraway (1988) speaks of “feminist 
objectivity” and states, “Feminist objectivity means quite 
simply situated knowledges” (p. 581). Haraway insists on 
“the embodied nature of all vision” and rejects that there 
is such a thing as “a conquering gaze from nowhere” 
(p. 581). It is the recognition of the contingency of knowl- 
edge claims that enables objectivity in science. Sandra 
Harding’s (1992) “strong objectivity” requires not just the 
recognition of the situatedness of knowledge claims but 
also the recognition that certain marginalized perspec- 
tives are epistemically privileged. 

If we accept a Longino-style account of scientific 
objectivity, peer review and post-publication criticism are 
important parts of the processes that secure the objec- 
tivity of science. Peer review serves not only to check 
that data seem right and that arguments are sound but 
also to introduce alternative viewpoints and thereby to 
minimize subjective biases. Similarly, post-publication 
reception in the form of replications, modifications, inte- 
gration with other results, and so on also contributes to 
refining new pieces of knowledge and shaping them into 
what is ultimately accepted (Longino, 1990, p. 69). The 
process of peer review is examined in more detail in the 
next section. 


4. Objectivity in Philosophy of Science in Practice 


In this section, we provide an example of how our concep- 
tion of scientific objectivity matters to current discussions 
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of how science should be organized. We focus on the 
question of how peer review should be organized, a ques- 
tion that has attracted a lot of attention in the past four 
decades. 

There are three common types of peer review: single- 
blind, double-blind, and open. The most commonly 
used is single-blind peer review, where the identity of the 
reviewer is concealed from the author, but the reviewer 
knows the identity of the author (Lee, Sugimoto, Zhang, 
& Cronin, 2013, p. 10; Ware, 2008, pp. 6-7). The main 
argument made for concealing the reviewer's identity 
to the author is to allow the reviewer to provide honest 
commentary without fear of repercussions. Double-blind 
peer review is also common, especially in the humani- 
ties and social sciences (Ware, 2008, pp. 6—7).’ The main 
argument made for concealing the identity of the author 
to the reviewer is to protect the author against social bias 
on the part of the reviewer. When peer review is open, 
finally, authors and reviewers know each other’s identi- 
ties, and reviews may even be put online. In most such 
systems, the name of the reviewer is made public (Lee 
et al., 2013, p. 10). 

Peer review can now be considered from two perspec- 
tives: (a) from the point of view that the objectivity of 
science is a function solely of the methods employed by 
the individual scientists and (b) from the point of view 
that the objectivity of science is also a function of how 
values and biases are distributed in the scientific com- 
munity and brought to bear on the process of knowledge 
creation. 

From the former perspective, concealing the identity 
of the author and/or reviewer contributes to the objectiv- 
ity of science. This is so because in practice, scientists are 
not as objective as legend has claimed. Scientists some- 
times fail to follow the scientific methods that ensure the 
objectivity of science on the former conception of objec- 
tivity. If reviewers always assessed papers with no regard 
to who the authors were, and if reviewers had no reason 
to fear retaliation from authors, there would be no need 
for concealing the identities of either author or reviewer. 
In sum, blinding the peer-review process contributes to 
maintaining the objectivity of science because scientists 
do not always follow scientific methods. 

From the latter perspective, the objectivity of science 
is a function of whether the perspectives of multiple 
individuals with different values and biases are brought 
to bear on the process of knowledge creation. Hence, 
concealing the identities of authors and/or reviewers 
may mask the distribution of values and biases and may 
therefore not support the effort to maintain the objec- 
tivity of science. Not knowing who the author is will 
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make it harder for the reviewer to uncover problematic 
background assumptions of the author. Likewise, not 
knowing who the reviewer is will make it harder for the 
author to uncover problematic background assumptions 
of the reviewer. Hence, on a Longino-style account of 
objectivity, disclosing author and reviewer identities can 
be an advantage. On the other hand, disclosing reviewer 
identities may work against the honest articulation of 
criticism because the reviewer fears retaliation from the 
author. Weighing such considerations is a complex task, 
and we shall not do so here. We merely want to empha- 
size that different conceptions of objectivity call for dif- 
ferent approaches to the question of how peer review 
should be organized. 

The last example we will consider involves open peer 
review of a certain kind. Some journals with open 
peer review make reviewer names and reviews publicly 
available. For example, the prominent medical journal 
BMJ publishes everything online: the reviewer comments 
and identities, the authors’ responses, and the different 
versions of the articles (Groves & Loder, 2014; Smith, 
1999, 2006). The aim of publishing everything online is 
to promote transparency and accountability. This trans- 
parency about reviewer comments and identities allows 
members of the community to assess the review while 
taking into account the theoretical and cultural perspec- 
tive it represents, that is, while having a way of recogniz- 
ing the background assumptions of the reviewer. In this 
way, reviews can be perspectives in the critical discus- 
sion of research on a par with other perspectives that are 
presented publicly (see also Lee et al., 2013, p. 11). These 
perspectives may even be especially helpful voices in the 
discussion since public peer review provides an incen- 
tive for reviewers to be both thorough and fair. Hence, 
if one conceives of objectivity as obtained at the level of 
scientific communities and as a function of the differ- 
ent perspectives that are brought to bear on the process 
of knowledge creation, this extreme kind of open peer 
review may help promote objectivity. 

How we conceive of scientific objectivity not only 
matters to the question of how peer review should be 
organized but also to the question of how to measure the 
reliability of peer review of the different kinds. In par- 
ticular, whether the level of agreement among review- 
ers is a good measure of the reliability of peer review 
depends on how we conceive of objectivity. Much the- 
orizing about peer review is based on the assumption 
that agreement among reviewers is something desir- 
able (cf. Lee, 2012, pp. 861-862). However, Kuhn’s and 
Longino’s accounts of science indicate that this is not 
necessarily the case. With reference to Kuhn, Lee (2012, 
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pp. 863-867) argues that reviewers can disagree in nor- 
matively appropriate ways about how to interpret and 
apply evaluative criteria such as soundness, significance, 
and novelty. To use Kuhn’s (1977, p. 325) own example, 
some scientists value originality more and are accord- 
ingly more prepared to take epistemic risks. They may 
reasonably have a lower evidential threshold for accept- 
ing a hypothesis. Furthermore, if Longino is right about 
the role of biases in the choices of individual scientists, a 
high level of inter-reviewer agreement could just as well 
be an indication that the referees have common biases. 
By the same token, low inter-reviewer agreement may be 
an indication that the scientific community in question 
supports effective criticism. Consequently, the fact that 
inter-reviewer agreement has been found to be low in 
empirical studies of peer review is not necessarily a cause 
for concern. 


5. Conclusion 


In this chapter, we have reviewed a number of accounts 
of scientific rationality and objectivity. Some philoso- 
phers consider scientific rationality and objectivity to 
be obtained at the level of individual scientists when 
the individual scientists follow a particular method 
for doing science. The chapter has presented divergent 
views on what that method consists in. Others argue 
that scientific rationality and objectivity are obtained at 
the level of communities of scientists. These accounts 
emphasize that individual scientists will be guided by 
different values, but that this in fact contributes to the 
rationality and objectivity of science at the level of sci- 
entific communities. We have presented different views 
on what this process looks like. 


Notes 


1. See Hoyningen-Huene (1987) for a detailed account of dif- 
ferent interpretations of the distinction between the context of 
discovery and the context of justification. 


2. For an example of a textbook aimed at university students 
that avoids conveying the idea that there is one scientific 
method, see Potochnik, Colombo, and Wright (2019). 


3. For detailed accounts of the various positions advanced 
during these decades, see, for example, Nickles (2017/2020), 
Losee (2004, 2005), or Newton-Smith (1981). 


4. For further details, see H. Andersen (2013). 


5. Longino examines numerous cases of research in which 
she claims biases have played an important role. Among other 
things, she studies cases of research in human evolution and 
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behavioral neuroendocrinology (see Longino, 1990, pp. 103- 
161). For a review of empirical studies of the role of bias in 
the peer review process in particular, see Lee, Sugimoto, Zhang, 
and Cronin (2013). 


6. Popper (1934/2014) similarly claimed that “the objectivity 
of scientific statements lies in the fact that they can be inter- 
subjectively tested” (p. 44). There is also an interesting similar- 
ity between Longino’s account of scientific objectivity and 
Jurgen Habermas’s account of how truth claims are justified. 
On Habermas’s (1973) account, truth claims are justified in a 
critical discussion among free and uncoerced participants. The 
reasons for the claim should in principle be acceptable to any 
reasonable person (see also Bohman & Rehg, 2007/2017). 


7. It is worth noting that double-blind peer review is some- 
times not possible: the content of a manuscript will some- 
times give away the author, which is presumably particularly 
true in highly specialized disciplines (L. E. Andersen, 2017, 
p. 178; Brown, 2007, p. 133; for further references, see Lee et al., 
2013, p. 10). 
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14.2 Rationality and the Value-Freedom of Science 


Anke Bueter 


Summary 


The value-freedom of science has traditionally been 
regarded as a presupposition of scientific rationality. 
However, during the past decades, in addition to numer- 
ous empirical counterexamples of value-laden science, 
systematic arguments have put the adequacy of value- 
freedom as an ideal into doubt. This chapter presents the 
most important debates on the ideal of value-freedom, 
which concern the epistemic impact of values in the 
discovery and justification of theories, the distinction 
between epistemic and nonepistemic values, and the 
argument from inductive risk. Taken together, these 
arguments call for new normative models of how to deal 
with values in science, which no longer equate value- 
laden science with bad science or irrationality. Rather, 
they suggest that scientific rationality is highly complex, 
since epistemic issues are interwoven with practical, 
sociopolitical, institutional, and ethical ones. 


1. Values and the Rationality of Science 


Science is often seen as an exemplarily rational endeavor, 
ideally generating knowledge untainted by extra-scientific 
interests, prejudices, or wishful thinking. As such, it can 
support rational beliefs about the world and serve as 
the basis for rational decision making in policy matters 
(e.g., concerning the mitigation of climate change). As 
Newton-Smith (1981) puts it, “The image that the scien- 
tific community likes to project of itself, and indeed the 
image that most of us accept of that community, is that of 
rationality par excellence” (p. 1). 

This rationality of science is secured by the methodi- 
cal production and testing of scientific knowledge 
that minimize random errors as well as systematic 
distortions—for instance, by value-laden perspectives. 
The value-freedom of science is therefore traditionally 
considered a precondition of its rationality.' Such a view 
is particularly plausible if one takes value judgments to 
be mere expressions of subjective preferences, incapable 


of rational justification. It is, however, equally compat- 
ible with all positions that consider value judgments 
to allow for rational discussion, yet not as the kind of 
judgment that can be supported by scientific methods. 
Essential to the ideal of value-free science is a separa- 
tion of the realms of is and ought: empirical conclusions 
do not follow from normative premises, and neither can 
empirical judgments legitimize normative conclusions. 


1.1. From Is to Ought 
Important predecessors of the current debate on science 
and values have often focused on this latter question of 
whether science can provide us with normative guidance 
in the ethical or political realm. David Hume is famous for 
criticizing deductions from is to ought (Hume, 1739/1975, 
p. 469). In the context of the developing social sciences 
in the 19th century, Max Weber has argued for the impor- 
tance of keeping science and politics separate. Expressing 
normative, political judgments in the role of (social) sci- 
entist, he says, is misguiding and only serves to immunize 
the respective judgments from criticism by other citizens 
(Weber, 1917/1973). The moral and rational thing to do 
would be to keep the roles of scientist and citizen dis- 
tinct and to clearly separate descriptive and normative 
judgments (on the relation of normative and descriptive 
theories, see also section 4 of the Introduction by Knauff 
& Spohn, this handbook). What science can do, accord- 
ing to Weber, is to study values and value systems. Their 
relations and coherency can be analyzed, and science can 
clarify whether and how certain political goals are empiri- 
cally realizable. For example, economics might inform us 
about the most effective ways to increase production, but 
it cannot tell us whether or not we should prioritize pro- 
ductivity over fair working conditions. Science may give 
us the means to realize certain practical ends, but it can- 
not justify these ends (cf. Weber, 1904). In other words: 
science should be politically neutral. 

This aim of neutrality has also been debated in the pos- 
itivism dispute between proponents of critical rationalism 
(Popper, Albert) and the Frankfurt school in sociology (e.g., 
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Adorno, Habermas) during the 1960s. The latter argued 
that a critique of the societal status quo is an important 
task of the social sciences, and the scientific justification 
of political goals is possible through historical material- 
ism. Critical rationalists criticized this approach for pur- 
porting normative assertions to be scientifically justified 
even though they are not testable by scientific methods. 
The Frankfurt school, they argued, borrowed the author- 
ity of science for their political positions but without sub- 
mitting them to the rigorous critical process that gives 
science its epistemic authority. Adorno and others replied 
that a political neutrality of science serves and perpetu- 
ates given power structures by failing to offer a critique 
and an alternative (cf. Adorno et al., 1969; see also Bueter, 
2012). 


1.2 From Ought to Is 

Since the 1980s, the value-freedom of science has yet 
again been subjected to intense debate. The current 
debate mostly focuses on the impact of value judgments 
on scientific theories and processes, asking whether such 
influences are avoidable—and, if not, how to deal with 
them. Here, the terms “value” and “value judgment” are 
often used in a very broad sense, in which “a value is 
something that is desirable or worthy of pursuit” (Elliott, 
2017, p. 11). This may include religious or social values 
but also economic interests, political goals, or individual 
preferences. Often, “value judgment” is characterized in 
a negative manner as comprising all judgments that can- 
not be justified by logic and empirical evidence alone. 
Prima facie empirical judgments can be “value-laden”; 
for instance, if they rely on an interpretation of the 
evidence from a certain interested perspective or result 
from biased methodological standards. 

Much research in philosophy and history of science as 
well as in science studies has brought forward examples 
of science that is value-laden in this sense. In particular, 
feminist scholars have exposed a multitude of cases where 
social prejudices related to gender have found their way 
into accepted scientific theories (cf., e.g., Keller, 1985; 
Kourany, 2010; Schiebinger, 1999). While such empiri- 
cal cases of value-ladenness were acknowledged, they 
have often been taken to underline, not undermine, the 
importance of value-freedom from a normative point of 
view. In other words, the ideal of value-freedom is often 
held on to: “Although complete value-freedom cannot 
be achieved, it ought to be a goal or ideal of science” 
(Shrader-Frechette, 1991, p. 44). 

This contemporary ideal of value-freedom” is usually 
based (implicitly or explicitly) on a distinction between 
the contexts of discovery, justification, and application.’ 
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There should be no compromise when it comes to the pro- 
posal that ideological factors be invited into the Context 
of Justification. ... We should make every attempt to keep 
politics and religion out of the laboratory. We may not 
always be successful, but that simply means that we should 
try harder, not that we should give up the attempt. (Koertge, 
2000, p. 53) 


“Context of discovery” usually refers to the generation 
of hypotheses. Whether these hypotheses are the result 
of a long thinking process, misogynism, or a nightmare 
about snakes is considered irrelevant for their epistemic 
status, which has to be assessed via rigorous and sys- 
tematic testing in the “context of justification.” Finally, 
empirically supported hypotheses and theories can be 
implemented in practice (“context of application”). Yet 
again, such an implementation is not considered a scien- 
tific decision. Instead, it is delegated to policy-makers and 
democratic deliberation, as it often requires the negotia- 
tion of different values and practical goals. 

The current ideal of value-freedom thus does not 
require science to be completely independent of any val- 
ues or interests; rather, these may impact the direction 
and application of research as well as decisions on the 
ethical admissibility of methods. However, the epistemic 
assessment (i.e., justification) of theories is supposed to 
be value-free. 

In the past decades, the feasibility of this normative 
ideal has been thoroughly questioned by systematic argu- 
ments that concern both the value-freedom of justifica- 
tion and its independency from values at play in discovery 
and application. In the following, I will present the most 
prominent debates on value-freedom: from the relevance 
of (values in) the context of discovery (section 2) and the 
distinction between epistemic and nonepistemic values 
(section 3) to the problem of inductive risk (section 4). 
The respective criticisms of the value-free ideal call for a 
refined understanding of scientific rationality that does 
not presuppose value-freedom. I will conclude with a brief 
outlook on resulting questions and emerging topics. 


2. Discovery, Pursuitworthiness, and Significance 


Restricting value-freedom to the testing and evalua- 
tion of theories presumes that this stage of the research 
process is epistemically independent from discovery 
and application. To put it another way, it assumes that 
there is a rational, scientific core of research, on the one 
hand, and more external stages not amenable to scien- 
tific rationality, on the other. Accordingly, the context 
of justification has often been treated as the sole proper 
subject of philosophy of science. 
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Against this, it has been argued that while there may 
not be an unambiguous logic of discovery, the generation 
of hypotheses and the selection of research questions are 
also not totally random or even irrational. Rather, they 
depend on ascriptions of significance that can be influ- 
enced by epistemic as well as sociopolitical, economic, 
or ethical considerations. 

Philip Kitcher argues that any assumption of a super- 
ordinate goal of science (such as truth, successful predic- 
tions, etc.) is incapable of efficiently directing research. 
Scientists are not equally interested in anything that may 
be true; rather, they search for significant truths (or signifi- 
cant predictions, explanations of significant phenomena, 
etc.). Such ascriptions of significance depend on more 
specific epistemic goals (e.g., making successful predic- 
tions about climate change or explaining the existence of 
our solar system). These specific epistemic goals are often 
informed by values. As Kitcher argues, even what seems 
epistemically significant (rather than practically impor- 
tant) today does so as result of a specific scientific history, 
which may have been influenced by value-laden con- 
cerns in earlier phases (Kitcher, 2001, chapters 6 and 7). 

Based on this, Kitcher develops the ideal of a well- 
ordered science, which broadens the focus of philosophy 
of science to include questions traditionally relegated to 
the context of discovery. Well-ordered science calls for 
political responsibility and accountability in the direc- 
tion of research, making it subject to a process of demo- 
cratic deliberation by representatives of different social 
interests (to be informed by impartial scientific experts; 
Kitcher, 2001, chapter 10; cf. also Kitcher, 2011). 

An important point to make here is that ascriptions of 
significance do not stop at the choice of a research ques- 
tion. To get from a specific question to a testable theory is 
a process that involves more than just the logical deduc- 
tion of empirical consequences from a certain hypoth- 
esis. It may involve questions of how exactly to frame the 
hypothesis, which variables to focus on, what concepts to 
employ,* and which empirical consequences of a hypoth- 
esis to test and how. Developing a testable theory, more- 
over, also depends on sufficient funding as well as the 
time and willingness to persevere in the face of setbacks. 

This has two important consequences: first, the dis- 
tinction between discovery and justification tends to 
make all of these important steps invisible, even though 
many of them are capable of rational evaluation. We 
could thus add a context of theory pursuit, where we judge 
theories not in terms of their eventual justification but 
their (epistemic and/or practical) prospects and fertility 
as well as initial plausibility (cf., e.g., Kordig, 1978; Nick- 
les, 1980; SeSelja & StraSer, 2014; Whitt, 1992). 
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[A] careful examination of scientific practice reveals that there 
are generally two quite different contexts within which theo- 
ries are evaluated. .. . Even if we had an adequate account of 
theory choice within the context of acceptance . . . we would 
still be very far from possessing a full account of rational 
appraisal. (Laudan, 1977, pp. 108-109) 


Second, several authors have argued that values in 
discovery and/or pursuit have impacts on the evaluation 
of theories, putting the epistemic independence of the 
context of justification in doubt. The general idea here is 
that ascriptions of significance impact which theories are 
developed and tested (and which not), as well as which 
data are generated and what is considered noise. This, 
in turn, impacts the assessment of existing theories. For 
example, Okruhlik (1994) argues that theory assessment 
is usually comparative: what is the best theory depends 
on the range of available theories. In a thoroughly sexist 
culture, though, “[n]on-sexist rivals will never even be 
generated. Hence, the theory which is selected by the 
canons of scientific appraisal will simply be the best of 
the sexist rivals; and the very content of science will be 
sexist” (Okruhlik, 1994, pp. 201-202). 

The choice between competing theories may thus be 
based on rational criteria and yet unable to screen out 
earlier social impacts—a point that Ohkruhlik takes to 
underline a need for more enhanced models of scientific 
rationality, which include the contexts of theory genera- 
tion and attend to the social embeddedness of science. A 
similar point has been made by Elliott and McKaughan 
(2009). Based on case studies from toxicology, they show 
how social values and interests impact the availabil- 
ity of theories and data—and, consequentially, theory 
assessment: 


The degree of evidential support for a theory clearly depends 
both on the array of available theories and on the set of data at 
hand. Therefore, to the extent that the nonepistemic values 
associated with discovery and pursuit influence the avail- 
able theory and data, they affect theory appraisal. (Elliott & 
McKaughan, 2009 p. 600) 


It is necessary to make choices about significance in 
scientific research. However, if all of these choices are 
guided by the same value-laden background assumptions, 
this can lead to bias in our results, even if these are tested 
methodically and meticulously. As Anderson (1995) 
explains, a partial truth can be thoroughly misleading. 
Bueter (2015) argues that values in discovery and pursuit 
can create blind spots such that alternative theories or 
data, which would undermine currently accepted theo- 
ries if they existed, are never even generated. For instance, 
this can happen due to widely shared sexist or racist 
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prejudices, or due to a domination of research fields by 
agents with particular interests, for example, when medi- 
cal research is driven by the pharmaceutical industry that 
aims to make profits and therefore does not explore non- 
pharmaceutical treatment alternatives (cf. Jukola, 2019). 


3. Values in Justification: The Social, the Epistemic, 
and the Nonepistemic 


Section 2 summarized arguments against a restriction of 
scientific rationality and value-freedom to the context 
of justification. For one, decisions concerning agenda- 
setting and pursuitworthiness are said to allow for rational 
discussion; for another, value-impacts on these decisions 
have been argued to affect theory assessment via the 
(non)existence of alternatives. However, a large part of 
research on science and values has focused on questions 
relating to justification. Prominently, it has been debated 
whether epistemic values can be distinguished from non- 
epistemic ones and, if so, whether the latter can or should 
be eliminated from theory assessment. 

The modern ideal of value-freedom is often taken to 
be essentially this: the evaluation of scientific contents 
is supposed to rely exclusively on epistemic criteria. 
Kuhn (1977) has been the first to call such criteria val- 
ues, thereby starting a still-ongoing debate on whether 
one can distinguish between epistemic (or cognitive)° 
and nonepistemic (noncognitive) values. Kuhn’s origi- 
nal list contained five such criteria that serve as indica- 
tors of good theories: accuracy (or empirical adequacy), 
requiring that a theory’s deducible consequences are 
in line with empirical observations; consistency (inter- 
nal, i.e., logical, as well as external, i.e., in relation to 
other currently accepted theories); scope; simplicity; and 
fruitfulness. 

Kuhn calls these criteria “values” because they do not 
provide algorithmic rules for theory choice, only guid- 
ance. They need to be interpreted and balanced against 
each other in specific research contexts. Moreover, their 
interpretation and weighing can be influenced by other 
factors such as the personality of a researcher (for a more 
detailed account of Kuhn’s position, see chapter 14.1 by 
Andersen & Andersen, this handbook). 

McMullin (1982) calls Kuhn’s values epistemic, arguing 
that they “promote the truth-like character of science” 
(p. 18), and distinguishes them from nonepistemic (e.g., 
religious or sociopolitical) values. Against this, Rooney 
(1992) points out the existence of different lists of such 
epistemic values, taking this variety to indicate a lack 
of clarity in the distinction between epistemic and non- 
epistemic. She further argues that this lack of clarity can 
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sometimes turn sociopolitical (e.g., androcentric) values 
into part of what it means to be epistemically successful. 

This resonates with Longino (1996), who argues that 
the function of epistemic values is to promote the goal(s) 
of science.° Such goals can be value-laden, for instance, 
when feminists aim to understand mechanisms of gen- 
der oppression or to produce nonsexist theories about 
women’s health. Thus, we could have research goals that 
are both feminist and epistemic—a status that translates 
accordingly to the values guiding theory evaluation. 
Starting from case studies on research with feminist 
goals, Longino creates an alternative list of such (femi- 
nist) epistemic values. She agrees that internal consistency 
and empirical adequacy are necessary features of a good 
theory.’ In contrast to Kuhn, however, she adds novelty, 
ontological heterogeneity, complexity of relation, applicabil- 
ity to human needs, and diffusion of power. Furthermore, 
she argues that the Kuhnian values can function as pro- 
moters of androcentric theories in particular research 
contexts (e.g., when simplicity serves as a justification 
of considering only men’s perspectives and concerns).* 

Others have defended the distinction between epis- 
temic and nonepistemic values. For instance, Ruphy 
(2006) proposes an empirical evaluation of epistemic 
values in terms of their contribution to empirically suc- 
cessful theories. Douglas (2013) argues that a more fine- 
grained classification of kinds of epistemic values can 
help both to reduce tensions between them and to sup- 
port their epistemic status. She does so by distinguishing 
epistemic values on two axes: (1) ideal desiderata versus 
minimal criteria and (2) values that apply to theories 
per se versus to theories in relation to evidence. Some, 
but not all, of the resulting four groups of epistemic val- 
ues are genuinely truth-assuring; others can be consid- 
ered strategic or pragmatic epistemic values in that they 
make it more likely (e.g., via testability) that flaws will 
be discovered (Douglas, 2013). 

Hugh Lacey argues that, when considering the role 
of epistemic and nonepistemic values in justification, 
it is necessary to distinguish between different kinds 
of attitudes with regard to the assessment of theories, 
which tend to be blurred under the common term of 
“accepting” a theory. According to him, a theory may be 
“adopted” (i.e., used to frame and direct future research), 
it can be “endorsed” (i.e., considered well enough sup- 
ported to put it into practical use), or it may be “impar- 
tially held of a set of phenomena” (i.e., considered so 
well established that it does not need further testing). 
In this third case, only epistemic values have legitimate 
roles to play (Lacey, 2017; cf. also Lacey, 1999). Rooney 
(2017) replies that Lacey’s argument is question-begging, 
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because it presupposes, rather than grounds, the distinc- 
tion between epistemic and nonepistemic values. 

Steel (2010) distinguishes between “intrinsic” and 
“extrinsic” epistemic values. Intrinsic epistemic values, 
such as empirical adequacy, are those that are necessary 
for the attainment of truth. Extrinsic epistemic values 
promote such an attainment only indirectly; examples 
here are testability or the ability to obtain funding. For 
Steel, nonepistemic values are values that are either irrel- 
evant or detrimental to the attainment of truth. 

Even if we assume the possibility of distinguishing 
epistemic from nonepistemic values though, further 
problems remain. In recent years, a controversy has 
emerged about the appropriate normative role of non- 
epistemic values in theory assessment, which focuses on 
the priority of epistemic over nonepistemic values. For 
instance, Steel’s account represents an “epistemic con- 
straints” approach: nonepistemic values are illegitimate 
if they interfere with or override the standards of ade- 
quate science expressed in extrinsic and intrinsic epis- 
temic values. On the other hand, they are allowed to 
play a role as “tie-breakers” if epistemic values are insuf- 
ficient to guide theory choice in particular cases. 

Steel (2017) contrasts such epistemic-constraints 
approaches with aim-oriented ones, which do not ascribe 
unambiguous priority to epistemic values. For instance, 
Elliott and McKaughan (2014) argue that the goals of 
inquiry are underdetermined if characterized as attain- 
ment of truth in general—rather, research aims for truths 
that are considered significant. The relevant goals here 
are therefore particular research goals that integrate prac- 
tical aims and concerns, which can be promoted by both 
epistemic and nonepistemic values. Nonepistemic val- 
ues may therefore have legitimate roles to play in theory 
assessment, as long as scientists are maximally transpar- 
ent about their goals and values. Relatedly, Brown (2013) 
argues that it is unnecessary to assign priority to epistemic 
values in order to avoid wishful thinking, as long as the 
value judgments involved are subjected to rational dis- 
cussion and are considered in principle revisable (cf. also 
Brown, 2017). A question for future research that arises 
here is how to determine the legitimacy of such epistemic- 
social goals (and thus values) and how exactly to avoid a 
corruption of research by political or economic interests. 


4. Application, the Consequences of Error, 
and Inductive Risk 


Another important strand in the debate on value- 
freedom focuses on the problem of inductive risk. The 
central question here is whether all responsibility for 
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decisions on applying research results can be exter- 
nalized or whether scientists themselves need to take 
into account potential practical consequences of their 
work—at least in the case of uncertainty. The argument 
from inductive risk says that practical consequences of 
mistaken scientific results are relevant to the setting 
of evidentiary standards, thereby rejecting the epis- 
temic independence of justification from the context of 
application. 

Rudner (1953) famously argued that in deciding to 
accept or reject any hypothesis, scientists always face 
two different kinds of potential errors: false-negative 
results (rejecting a true hypothesis) versus false-positive 
results (accepting a false hypothesis). This gives rise to 
the question of how much evidence is enough to decide 
upon the acceptance or rejection of individual hypoth- 
eses in the light of error risks. Consequently, Rudner 
argues, scientists need to refer to value judgments con- 
cerning the potential consequences of different kinds 
of errors. For instance, if the acceptance of a hypoth- 
esis about the safety of a new drug will likely lead to 
its approval and use in a large population, a false posi- 
tive may lead to many patients experiencing severe side 
effects, while a false negative would mean the loss of 
a potential treatment to them and of financial profits 
to the pharmaceutical company. Which kind of error is 
considered worse in a specific situation depends on non- 
epistemic value judgments and impacts the epistemic 
standards for acceptance (e.g., if it is most important to 
us to avoid a false-positive result, these standards will be 
set very high). 

Rudner (1953) concludes that this means that the sci- 
entist qua scientist necessarily makes value judgments. 
Against this, Levi (1962) has argued that Rudner con- 
flates accepting a hypothesis in terms of its epistemic 
validity and thus believing in its truth, with acting based 
on such a belief. Relatedly, Jeffrey (1956) has objected 
that scientists often cannot foresee potential practical 
applications of their work and thus consequences of 
error. Therefore, they can hardly decide upon the accep- 
tance/rejection of a hypothesis based on values referring 
to these consequences. Instead, Jeffrey proposes that 
scientists should not make such decisions but confine 
themselves to ascribing probabilities to hypotheses— 
leaving it to policy-makers to decide how high a prob- 
ability is high enough in a particular situation. Pointing 
out the problems with leaving such value judgments 
to scientists rather than democratic decision making, 
Betz (2013) similarly argues that scientists should not 
accept or reject plain hypotheses but rather “hedged 
hypotheses,” which articulate the amount and kind of 
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uncertainty involved in making a certain claim (for a 
response, see John, 2015a). 

In one of the most influential articles in the debate, 
Douglas (2000) has both resuscitated Rudner’s argument 
and strengthened it. First, she confines the argument to 
policy-relevant research, where practical consequences 
of scientists’ assertions often are predictable. Second, 
she shows how trade-offs between the probabilities of 
false positives versus false negatives occur not only at 
the point of a hypothesis’s final evaluation but at several 
methodological stages before. For instance, such trade- 
offs relate to the setting of a significance level, the gath- 
ering and interpretation of data, or the interpretation of 
results. This makes it more difficult to externalize all the 
decisions facing inductive risk and implies a thorough 
value-ladenness of science. For Douglas, this amounts to 
an ethical obligation of scientists to consider the poten- 
tial practical consequences of errors. For her, the respec- 
tive ethical considerations should have an impact on 
evidential standards in science. She distinguishes such 
legitimate indirect value influences from illegitimate 
ones, where nonepistemic values directly impact the 
rejection/acceptance of a hypothesis: 


Two clear roles for values in reasoning appear here, one 
legitimate and one not. The values can act as reasons in 
themselves to accept a claim, providing direct motivation 
for the adoption of a theory. Or, the values can act to weigh 
the importance of uncertainty about the claim, helping 
to decide what should count as sufficient evidence for the 
claim. (Douglas, 2009, p. 96) 


Douglas’s work has been the beginning of an ongoing 
and very lively debate about inductive risk in science. 
For one, the argument of inductive risk has been applied 
in a variety of case studies from medicine over climate 
science to theoretical physics (cf., e.g., Elliott & Rich- 
ards, 2017). For another, its soundness, its scope, and its 
impact on the ideal of value-free science continue to be 
discussed (cf., e.g., Biddle, 2013; de Melo-Martin & Inte- 
mann, 2016; Hicks, 2018). Biddle (2016) as well as Biddle 
and Kukla (2017) have recently called for, and started 
on, a more fine-grained differentiation of kinds of risk 
involved in inductive risk. 

Douglas’s normative answer—distinguishing between 
indirect value influences on evidential standards as legit- 
imate and direct influences on acceptance decisions as 
illegitimate—has also been called into question, in terms 
of both its conceptual clarity and its normative suffi- 
ciency (e.g., Elliott, 2011). In particular, indirect influ- 
ences can also sometimes be problematic, for example, if 
they overly tip the balance between error probabilities in 
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favor of sponsor interests (cf. Wilholt, 2009, for an exam- 
ple). Wilholt, moreover, provides an alternative proposal 
for identifying biased research, which relies on the role 
of conventions in the scientific community. Such con- 
ventions serve as a means to standardize methodological 
decisions that can indirectly affect acceptance decisions. 
Deviations from such commonly employed standards 
can thus make value influences visible. In addition, 
Wilholt argues that the distribution of error probabili- 
ties is not only a problem in cases of applied research 
with foreseeable social consequences. It is rather an in- 
principle epistemological issue, since false-positive or 
false-negative results differ in relevance and importance 
even if we consider only epistemic utilities, such as the 
overall importance of a certain hypothesis for our belief 
systems and theories (Wilholt, 2009, 2013). 


5. Conclusion 


To sum up, the value-freedom of science has been the 
subject of intense debate in the past decades—both in 
terms of numerous empirical counterexamples and 
regarding its adequacy as a normative ideal. At the same 
time, most philosophers do not equate value-ladenness 
with bad science (anymore). Instead, they aim for more 
sophisticated approaches, supposed to be able to distin- 
guish between illegitimate and legitimate value influ- 
ences; for example, in terms of kinds of values (i.e., 
epistemic vs. nonepistemic), indirect or direct roles of 
values, or stages of scientific research in which they may 
or may not play a role. While this debate on normative 
alternatives is still ongoing, the emerging consensus 
seems to be both that the traditional value-free ideal is 
doomed—and that this does not imply that science lacks 
rationality. Rather, it indicates that the kind of rational- 
ity involved is highly complex, since epistemic issues 
are thoroughly interwoven with practical, sociopolitical, 
institutional, and ethical concerns. 

Since one of the purported functions of the ideal of 
value-free science was to provide the public with factual, 
untainted information to be used in democratic delib- 
erations on practical concerns, an emerging topic now is 
how this new image of science relates to issues of public 
trust (see also chapter 14.3 by Bromme & Gierth, this 
handbook). In particular, if science is shot through with 
ethical and social values, the question arises whose val- 
ues these are, how they are to be legitimated, and how 
this can be compatible with rational trust in the results of 
such science. Furthermore, this leads to the question of 
how to communicate to the public the role of values in 
scientific research. While, for instance, McKaughan and 
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Elliott (2013) have pleaded for transparency here, John 
(2015b) warns against a further erosion of public trust 
caused by such transparency. To conclude, the question 
at this point in time no longer seems to be whether sci- 
ence is value-free or whether it must be so in order to be 
rational—but rather how to deal with value-laden sci- 
ence in a sensible manner, not least on the institutional 
level of organizing scientific research and science—public 
relations. 


Notes 


1. It has also usually been treated as a necessary, yet not suf- 
ficient, condition for objectivity. For a more detailed look at 
objectivity, see chapter 14.1 by Andersen and Andersen (this 
handbook). 


2. Regarding different elements and historical versions of the 
value-free ideal, see, for example, Proctor (1991). 


3. For a closer analysis and discussion of the context distinc- 
tion, see Schickore and Steinle (2006). I am referring to it here 
not in order to support it but because it has had a considerable 
impact on the ideal of value-freedom and constitutes a useful 
means to organize different discussions in the science-and- 
values debate. 


4. Another discussion I will not go into here for reasons of 
space concerns the semantic value-ladenness of scientific con- 
cepts, classifications, and terminologies (cf., e.g., Dupré, 2007). 


5. The terminology differs here. For the sake of readability, I 
will use “epistemic” throughout in the following. 


6. Helen Longino’s work has played a very decisive role in 
debates on values in science; this holds in particular for her 
account of social objectivity, which does not presume value- 
freedom. Rather, social objectivity requires scientific communi- 
ties to be organized in a way that enhances the efficiency of 
critical discourse (Longino, 1990; for more details, see chap- 
ter 14.1 by Andersen & Andersen, this handbook). 


7. Similar to Kuhn, Longino takes consistency and empirical 
adequacy to be insufficient to determine theory choice. In both 
cases, this assumption is based on moderate versions of the 
underdetermination thesis. Kuhn argues that rival theories are 
often not empirically equivalent, yet both more or less empiri- 
cally adequate, differing in what areas of evidence they match 
better (Kuhn, 1977, p. 323). Longino proceeds on the basis of 
an underdetermination thesis regarding evidential relations. 
As she argues, the relation between theoretical hypotheses and 
(significant) empirical consequences is mediated by substantial 
and methodological background assumptions, which are pos- 
sibly value-laden (Longino, 1990, pp. 40-48). 


8. Such a diversity in values guiding theory assessment might 
be taken to undermine the possibility of rational discourse 
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in the case of dissent. Carrier (2013) argues that pluralism 
extending to the epistemic values used in theory assessment is 
unproblematic if the scientific community shares a fundamen- 
tal “epistemic attitude” toward the goals of research, which 
enables a joint striving for consensus and rational discussion. 
This epistemic attitude is not implemented at the level of stan- 
dards for theory assessment but at the level of procedural rules 
for debating knowledge claims. 
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14.3 Rationality and the Public Understanding of Science 
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Summary 


Science’ and rationality are strongly linked. Therefore, 
the public understanding of science’ is crucial for ratio- 
nality as an everyday mode of reasoning. However, the 
knowledge provided by science is distinct from every- 
day thinking to a large degree, and at least nonscientists’ 
understanding of it is limited. This chapter will provide 
an overview of the tension between the significance of 
science as a source for rational reasoning and the bound- 
edness of nonscientists’ understanding of science. It will 
be argued that trust is a key concept for analyzing how 
people cope with this boundedness. It will be elaborated 
why asking Whom to believe? is a rational way for citizens 
to reason about the truth of scientific knowledge claims. 
Then, motivated reasoning and knowledge as psychologi- 
cal conditions for judging what to believe and whom to 
trust will be discussed. Finally, the question about ratio- 
nality within the public’s understanding of science will 
be placed into the context of controversies between nor- 
mative and descriptive accounts of rationality. 


1. The Nexus between Science and Rationality 


Rational inferences as well as rational beliefs are deemed 
valid because their validity has been established by sci- 
entific methods. Why is it justified to evaluate a Bayesian 
inference about the probability of an event as a rational 
inference? Why is it justified to evaluate the belief that 
objects move at a constant speed as a rational belief? In 
both cases, these justifications have been established by 
the scientific community as scientific knowledge (Bayes’ 
theorem, Newton’s first law). Rational decisions about 
the acceptance or rejection of such inferences and beliefs 
are therefore embedded in the acceptance of scientific 
background knowledge. In this regard, science is a 
source for establishing normatively what is “rational.” 
Conversely, the production and justification of knowl- 
edge is accepted as scientific only if it can be described 
as rational. 


The relationship between science and rationality is 
strong, but they are not equivalent concepts: it is possible 
to make rational inferences without reference to scientific 
knowledge. Conversely, rationality is necessary for science 
but not a sufficient condition for science.’ This can be illus- 
trated by the problem of demarcating between science and 
pseudoscience (e.g., creationism, climate change denial- 
ism, homeopathy, flat-earth theories, chem-trail theories). 
Pseudoscientists claim to adhere to scientific criteria of 
truth, they provide supposedly supporting evidence, and 
they mimic scientific discourse, especially when attack- 
ing real science. In this regard, pseudoscience is different 
from conspiracy theories and mere superstition, although 
the boundaries between these kinds of belief frameworks 
are fuzzy, and belief in one is correlated with belief in the 
other. There is no set of precise necessary and sufficient 
criteria for the demarcation between pseudoscience and 
science, but it is possible to tell them apart because pseu- 
dosciences typically ignore some of the criteria of science, 
for example, the need for coherence among its theoreti- 
cal propositions (Law, 2020; Mahner, 2013). The validity 
of scientific inferences is not guaranteed solely by their 
adherence to normative rules of rational reasoning. All 
rational reasoning refers to some background knowledge 
about the world (Chater & Oaksford, 2012). The problem 
of demarcating between pseudoscience and science illus- 
trates that “rationality” is not just an issue of adhering to 
norms of reasoning but also an issue of adopting shared 
assumptions about the world. For example, the inference 
that the dilution of a pharmaceutical agent increases its 
potency (a basic tenet of homeopathy) could be deemed 
rational as long as it is consistent with background beliefs 
about a world including the capacities of substances to 
store and to transmit health-related information irrespec- 
tive of molecular concentrations. Hence, there could be 
a local rationality within the worldview of pseudoscien- 
tists. Nevertheless, in a more global sense it is not ratio- 
nal, because these background beliefs can be justified 
only by selectively ignoring contradicting evidence and 
arguments. 
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The validity of this background knowledge about 
the world does not have to be established as scientific 
knowledge, but with regard to many domains of our 
lives, it can* only be established as scientific knowl- 
edge. The strong nexus between science and rational- 
ity follows from the fact that the understanding of the 
actual structure of our environment (the natural and 
the social world) is formed by science. Modern socie- 
ties rely on all kinds of artifacts (technological as well as 
social) that have been developed by means of scientific 
knowledge. Moving successfully in this world requires 
at least a certain degree of understanding and accep- 
tance of this scientific knowledge, although it is possible 
(for some people within some contexts) to live without 
such acceptance, as the case of pseudoscience believers 
shows. In this regard, rationality also rests on the (pub- 
lic) acceptance of what can be taken as “true” (in the 
sense of scientifically justified) worldviews, not only on 
the local adherence to certain rules of rational reason- 
ing. This does not mean that such a public acceptance 
can be taken for granted; it has to be maintained con- 
tinually, for example, via schooling and science com- 
munication. In other words, a “public understanding of 
science” (PUS) (in a sense that will be detailed below in 
this chapter) is crucial for rationality. 

Because of this nexus, analyzing the public under- 
standing of science could contribute to understanding 
rationality and its limitations among the general pub- 
lic. There are roughly three strands of research provid- 
ing empirical insight into conditions and processes of 
citizens’ understanding of science: (a) research on the 
emergence and fostering of scientific literacy (SL), typi- 
cally focused on children and young adults in formal 
learning contexts; (b) studies about the processing and 
acceptance of scientific beliefs, typically focused on 
adults encountering scientific claims via the media; and 
(c) studies about the personal stances toward science 
held by different segments of the general public, typi- 
cally based on representative surveys. 


2. What Is Meant by Public Understanding of Science? In 
Which Sense Is This Understanding Bounded? 


Scientific literacy (SL) and the public’s bounded® under- 
standing of science are related but distinct concepts. 
According to a recent definition, covering widely shared 
notions of SL, a scientifically literate person has the 
competencies to “explain phenomena scientifically— 
recognise, offer and evaluate explanations for a range of 
natural and technological phenomena. Evaluate and design 
scientific enquiry—describe and appraise scientific 
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investigations and propose ways of addressing questions 
scientifically. Interpret data and evidence scientifically— 
analyse and evaluate data, claims and arguments in a 
variety of representations and draw appropriate scien- 
tific conclusions” (OECD, 2016, p. 13, emphasis added). 
Based on this definition, international studies within 
the PISA project revealed a large variance in the degree 
of SL, between countries as well as between individuals. 
In this regard, it can be assumed that—at least for large 
parts of the general public—the individual understand- 
ing of science is limited. 

However, there are further limitations. SL typically 
refers to knowledge items and to the kinds of scientific 
reasoning that are taught in schools (DeBoer, 2000). 
These knowledge items are in principle understandable 
for the general public without specialized science train- 
ing. Furthermore, they contain knowledge that has been 
established as certain, at least within the pertinent sci- 
entific community. Understandability and certainty are 
conditions for including a piece of knowledge in the 
school curriculum. Therefore, only a narrow selection of 
all the scientific knowledge and all the kinds of scientific 
reasoning (Kind & Osborne, 2017) can be included. In 
contrast, everyday encounters with natural and techno- 
logical phenomena are confined neither to understand- 
able nor to certain scientific knowledge. Parents reason 
about the actual or supposed scientific evidence put for- 
ward in favor of certain educational practices, patients 
reason about contradictions between their doctors’ 
arguments and what they read on the Internet, and citi- 
zens reason about contradicting arguments concerning 
the feasibility of carbon storage as a means of reducing 
global warming. In this regard, public understanding of 
science refers to a much broader spectrum of citizens’ 
encounters with science than the concept of SL. Fur- 
thermore, which topics emerge within the public dis- 
course is not limited to topics that are understandable 
or certain. Which topics are receiving attention by the 
general public reflects practical problems on a personal 
(Feinstein, 2011) as well as a societal level and—in a 
mutual relationship—the media coverage of certain top- 
ics (National Academies of Sciences, Engineering, and 
Medicine, 2017). Thus, neither understandability nor 
certainty of knowledge is a limiting factor for which top- 
ics emerge within the public discourse about science. 

There are further epistemological as well as psycholog- 
ical reasons to conceive the boundaries of understanding 
science as a constraint on the public’s encounters with 
science. One epistemological reason is the complex- 
ity of scientific knowledge. This refers to the possibly 
unlimited depth of scientific explanations (Keil, 2010). 
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Additionally, there are the epistemic, social, and tech- 
nological complexities of the methods and procedures 
necessary to establish a scientific knowledge claim as a 
true belief about the world. Nowadays, the production 
of scientific evidence is based on an elaborate division of 
cognitive labor, which is mirrored in the disciplinary struc- 
ture of science as well as in the volatility of this structure. 

A further epistemological reason refers to the qualita- 
tive differences between the objects, models, and meth- 
ods of science, which refer to a certain piece of the natural 
or the social world and the everyday experiences with 
this piece. The emergence of science in modern history 
has also been a history of separating a scientific under- 
standing from an everyday understanding of the world 
(Wolpert, 1992). Many entities of science could not be 
experienced with the unaided senses. Even concepts 
that are well established in our everyday reasoning, like 
“bacterium,” “virus,” or “gene,” refer to entities that we 
cannot experience without technology. Some processes 
can only be modeled mathematically, for example, in 
nuclear physics. Others extend the number and kinds of 
dimensions that make up our everyday experience of the 
world. Such models can be linked to everyday reasoning 
only by way of analogies (Boudry, Vlerick, & Edis, 2020). 

Research on the psychological differences between 
scientific and everyday knowledge has found systematic 
misconceptions (from a normative, i.e., scientific, point 
of view) in several domains of knowledge (Shtulman, 
2017). Such misconceptions are robust, and even after 
science instruction, they still coexist with elements of 
scientifically justified knowledge. Two prominent fea- 
tures of everyday thinking illustrate that these differ- 
ences are not due to a void within individual knowledge 
structures (and should thus not be explained in terms of 
a deficit model) but due to the stability and local coher- 
ence of the already existing knowledge. Both features 
refer to ontological differences between the kinds of 
categories used in scientific modeling and in everyday 
thinking: everyday reasoning prefers explanations in 
terms of substances and their features, while scientific 
thinking requires an understanding of relationships. For 
example, concepts like mass, energy, and heat are—from 
a physics point of view—concepts about relationships, 
while in everyday reasoning, they are dealt with as fea- 
tures of objects (Slotta & Chi, 2006). Everyday reason- 
ing accepts “essences” as sufficient causes for explaining 
phenomena. In this vein, genetic explanations are popu- 
lar in everyday reasoning about developmental and 
behavioral patterns of humans because they causally 
link the essence-like genes with these outcomes (Heine, 
Dar-Nimrod, Cheung, & Proulx, 2017). 
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This boundedness of science understanding is not 
fixed, neither historically nor individually. Specialist 
knowledge can become part of the school curriculum, 
as has been the case with concepts from nuclear physics 
or molecular genetics. These are examples for historic 
shifts of the boundary between specialist knowledge 
and public knowledge, at least for well-educated citi- 
zens. Appropriate instructional environments (e.g., sci- 
ence education in schools, lab experiences in science 
museums) can increase individual scientific literacy. 
Well-designed information about health issues, nuclear 
physics, or climate change is available. It is possible to 
increase scientific knowledge about topics of public 
interest. This is what strategic public-understanding-of- 
science campaigns are aiming for. 

Nevertheless, this does not nullify the constraining 
effect of the previously outlined boundaries of under- 
standing science for nonscientists. Such boundaries 
matter especially when it comes to conflicting scientific 
knowledge claims. When citizens look up health-related 
information on the Internet (e.g., about the relationship 
between nutrition, body weight, and illnesses), they will 
come across claims that do not align with each other 
or with claims made by their medical specialist. Even 
when citizens look up evidence for the existence of cli- 
mate change, they will come across contradictory voices. 
Hence, there are not only conflicts between claims put 
forward by different scientists but also—and practically 
more important—conflicts between the claims of scien- 
tists and those of actors who would not be conceived 
as members of the scientific community, for example, 
deliberate science deniers, quacks, preachers of esoteric 
belief systems, conspiracy theorists, and “merchants of 
doubt” (Oreskes & Conway, 2010). 

Often, the claims themselves and the reasoning by 
which they follow from the evidence are understandable. 
When they are presented well, they can be understood 
by nonexperts, even with a low level of SL. However, 
when reasoning about which of the conflicting knowl- 
edge claims is true and which is false, the abovemen- 
tioned boundaries of public understanding matter. In a 
British survey (Castell et al., 2014), 70% of respondents 
agreed that there is so much conflicting information 
that it is hard to know what to believe. 


3. Whom to Believe? The Role of Trust in Reasoning 
about Scientific Knowledge Claims 


When coming across conflicting claims, one could evalu- 
ate these claims as more or less plausible and then accept 
one as a subjective belief. But with regard to many topics, 
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most members of the general public are not able to decide 
by virtue of their own understanding of the relevant topic 
which scientific knowledge claim should be adopted as a 
true belief (in the sense of being justified scientifically). 

Another way of coping with the encounter of com- 
peting knowledge claims could be a shift from the 
question of the plausibility of the content (“What to 
believe?”) toward the question of the trustworthiness 
of its respective sources (“Whom to believe?”). The lat- 
ter question is both logically and empirically quite dif- 
ferent from the former. Trust is a kind of willingness 
to depend on others. Whenever people are dependent 
on other agents (persons or organizations), and when- 
ever they are willing to accept the risks that come along 
with this dependency, they (the “trustors”; Bl6baum, 
2016) put trust into these agents (the “trustees”). In the 
context of science, the good that the trustee provides to 
the trustor is “knowledge,” and the risk to the trustor 
is his or her vulnerability to a lack of truth or valid- 
ity of that knowledge (Hendriks, Kienhues, & Bromme, 
2016). This is called epistemic trust (Sperber et al., 2010). 
Judging the validity of a claim on the basis of an ascrip- 
tion of trustworthiness to the source has been called 
the “argument from authority” or “argumentum ad 
verecundiam.” Its logical form is “According to person 
A, who is an expert on the issue of Y, Y is true. Therefore, 
Y is true.” This is, from a classical logical perspective, a 
fallacious inference (Cummings, 2014), and it seems to 
be at odds with scientific reasoning and argumentation. 
A core principle of modern science is to know the truth 
instead of just trusting what you are told. “Historically, 
this individualistic stance could be seen as a reaction 
against the pervasive role in Scholasticism of arguments 
from authority” (Sperber et al., 2010, p. 361). Only a 
minority of philosophers and historians of science have 
emphasized the role of trust in science, and the ones 
who do focus mostly on how it relates to doing science 
and only to a lesser degree on science communication 
with the public (Hardwig, 1991; Irzik & Kurtulmus, 
2019; Wilholt, 2013). However, the main argument for 
trust within science, the division of cognitive labor,° 
is the same reason why trust is essential in the general 
public’s encounter with science (Bromme & Goldman, 
2014; Levy, 2019). 

Within informal reasoning, the argument from author- 
ity is a legitimate inference (Cummings, 2014; Keren, 
2018). It is a shift of the topic of rational reasoning (from 
“What to believe?” to “Whom to believe?”), not the 
denial of rationality. Reasoning about “Whom to believe?” 
requires thinking about science, not as a cognitive struc- 
ture, but as a social system. 
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Research has revealed that judgments about trust 
in scientists involve three dimensions (although the 
terminology is not consistent), which are in line with 
trust research in other domains: expertise, integrity, and 
benevolence (Cummings, 2015; Hendriks, Kienhues, & 
Bromme, 2015; Peters, Covello, & McCallum, 1997). 
Expertise refers to someone’s amount of knowledge and 
skill. The dimension of expertise also encompasses the 
aspect of pertinence (Bromme & Thomm, 2016): the 
person must have the relevant expertise. Integrity refers 
to adherence to the rules of the profession and—in the 
context of science—to the intention to produce “true” 
(unbiased) knowledge. Benevolence refers to taking into 
account the interests of the trustor or (more generally) 
the goods of the scientist’s work for society. Importantly, 
people recognize that a scientist can have his or her own 
interests in mind and still be benevolent (Critchley, 
2008). 

When do people shift from a “What to believe?” 
toward a “Whom to believe?” strategy? Recent research 
on the conditions under which people pay attention to 
source information to assess the quality of knowledge 
claims (Braten, Stadtler, & Salmeron, 2017; Bromme, 
Stadtler, & Scharrer, 2018) has corroborated two main 
results: first, there is no spontaneous attention to sources, 
and thus perceived content quality is unaffected by 
source information. Second, people only actively pro- 
cess source information when encountering some kind 
of discrepancy. Such discrepancies can be perceived when 
claims are inconsistent, either within the same piece of 
information or across two or more different pieces of infor- 
mation or between the piece of information and prior beliefs 
held by the recipient. When such discrepancies occur 
and people turn to the source information to determine 
which claim is correct, both types of judgment inform 
each other: on the one hand, people make inferences 
about the trustworthiness of sources based on their 
willingness to accept the messages provided by these 
sources. On the other hand, assumptions about a source 
affect judgments about the plausibility of their claims 
(Hahn, Harris, & Corner, 2009; Lombardi, Seyranian, & 
Sinatra, 2014; Thomm & Bromme, 2016). 

There is a preference for immediate judgments about 
the plausibility or believability of claims; people seem 
to be epistemic individualists (Levy, 2019). There are dif- 
ferent approaches to explaining this default preference. 
It could be explained as a case of a general overconfidence 
regarding one’s own knowledge (Atir, Rosenzweig, & Dun- 
ning, 2015), as a case of a more specific illusion of explan- 
atory depth—confined to the assumed understanding of 
causal mechanisms (Mills & Keil, 2004)—or as a kind of 
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fluency effect, called the easiness effect. Scharrer, Stadtler, 
and Bromme (2014) asked laypersons to read short texts 
containing medicine-related knowledge claims. In all 
conditions, judging the validity of the medical claims 
was clearly beyond laypersons’ medical or biological 
knowledge, but the texts varied regarding their lexical 
simplicity (easiness). Subjects who read the easier (but 
still conceptually very difficult) texts reported a lesser 
need for further expert advice to judge the credibility 
of the claim. However, this easiness effect was mitigated 
when subjects were explicitly told about the epistemic 
complexity of the medical topic. Awareness of the epis- 
temic complexity matters for people’s beliefs about the 
explanatory power of science. It seems to be a condition 
for deference to science as an epistemic authority. 


4. Conditions for Judging What to Believe and Whom 
to Trust: Motivated Reasoning and Knowledge 


Recent research on the public understanding of science— 
especially within the abovementioned strands (b) (stud- 
ies about the processing of scientific beliefs) and (c) 
(studies about the personal stances toward science held 
by different segments of the general public)—has put a 
strong emphasis on the public’s personal acceptance of 
specific scientific claims, on their general appreciation of 
science, and on the public’s trust in science and scien- 
tists (Hendriks et al., 2016; Rutjens, Heine, Sutton, & van 
Harreveld, 2017). 

This research focuses more on attitudes about science 
rather than on the understanding of science in the sense of 
the abovementioned research strand (a), regarding scien- 
tific literacy. Nevertheless, since attitudes refer to beliefs 
about the world, they matter for reasoning about the 
world and hence for rationality. This shift in focus has 
been fueled by evidence about the weak and sometimes 
inverse relationship between knowledge about science and 
acceptance of scientific knowledge as valid beliefs. Addi- 
tionally, it has been fueled by findings pointing to the 
impact of attitudes, religious and political beliefs, group 
norms, personal identity, self-concepts, and personal 
goals on the processing of science information. Recent 
research on public understanding of science conceptual- 
izes such processing as motivated reasoning (Kunda, 1990), 
subsuming these personal goals and beliefs as “motiva- 
tions” for (biased) reasoning in favor of results that align 
with these motivations. Both the impact of knowledge 
and that of such motivations on reasoning are crucial for 
rationality and will hence be discussed in the following. 

Motivations matter for the selection of information as 
well as for its subsequent processing. Selective exposure 
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and selective avoidance, respectively, can lead to so- 
called echo chambers, in which people only hear what 
they already believe. Regarding the relative importance of 
these processes, people’s preference for belief-confirming 
information seems to be stronger than their avoidance 
of belief-disconfirming information (Garrett, 2009). 
Motivated reasoning’ in general, and so-called defensive 
motivations in particular, could lead people to discount 
belief-disconfirming evidence by devaluating science in 
general or by doubting its ability to provide true knowl- 
edge about critical issues (Munro, 2010). For example, 
computer gamers disparage research on negative con- 
sequences of playing violent video games due to anger 
about the goal of this research and the stigmatization of 
gamers that they perceive to go along with it. This deval- 
uation of the critical research findings generalizes to the 
entire research about video games (Nauroth, Gollwitzer, 
Bender, & Rothmund, 2014). Indeed, being a member of 
a certain social group, be it gamers, parents, or support- 
ers of a certain political party, often provides motivations 
that affect reasoning. In the United States, Republicans 
and Democrats differed in their opinions on 16 sociosci- 
entific issues, from mandatory childhood vaccinations to 
climate change and the teaching of evolutionary theory 
(Blank & Shaw, 2015). In fact, that Democrats are more 
likely to accept the existence of anthropogenic global 
warming is the most extensively studied case of how 
political opinions affect science attitudes and trust. 
Typically, the main focus of such controversies is 
rather political or cultural than scientific (Who will have 
to pay, in any sense, for human-caused climate change?), 
but citizens’ beliefs about the validity of the implied sci- 
entific knowledge claims (e.g., about effects of carbon 
emissions on the mean temperature) are affected by 
these debates and by the political and ideological land- 
scape of a society (Gauchat, 2012). For example, Lewan- 
dowsky and Oberauer (2016) have shown that beliefs in 
the importance of free markets are negatively correlated 
with the acceptance of human agency as the cause of 
climate change. Some researchers on PUS argue that the 
affiliation with a certain culture (a set of related political, 
ideological, and religious stances) is the main predictor 
for the appreciation of science and for the acceptance 
of scientific knowledge claims as valid (Kahan, Jenkins- 
Smith, & Braman, 2011). It is questionable if these pat- 
terns of beliefs and affiliations to social groups within 
a society are properly described as cultures (van der Lin- 
den, 2016), but there is enough evidence that such belief 
systems impact directly on the acceptance of scientific 
claims as true beliefs and on trust in science. Exam- 
ples for the impact of belief systems are framing effects 
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in reasoning. In a study in the United States, by sim- 
ply substituting the term “climate change” for “global 
warming” on a survey questionnaire, Republicans were 
rendered more willing to indicate agreement with its 
existence (Schuldt, Roh, & Schwarz, 2015). 

These belief systems and reasoning directed by them 
also modify the role of knowledge for the public’s trust in 
science as well as for the acceptance of specific scientific 
claims. Early strategic endeavors for an improvement of 
the public understanding of science followed a coarse 
enlightenment heuristic: “If the public understands sci- 
ence, it will believe in science and support scientists.” 
However, it rapidly became evident that this heuristic is 
flawed. There is a positive but small correlation between 
general knowledge about science and general trust in sci- 
ence (Allum, Sturgis, Tabourazi, & Brunton-Smith, 2008; 
Shi, Visschers, Siegrist, & Arvai, 2016). In contrast, when 
it comes to topics that are linked to political, religious, 
or cultural debates, the relationship between specific 
knowledge about the critical topic and trust in scientific 
evidence about this topic is mediated by the relevant 
belief systems: Kahan et al. (2012) found higher levels 
of science knowledge being related to increased polar- 
ization between Republicans and Democrats in regard 
to global warming belief. Similarly, accuracy of vaccina- 
tion beliefs was most polarized among well-educated 
Americans (Joslyn & Sylvester, 2019). Broader science 
knowledge could enable people holding extreme beliefs 
(e.g., climate change deniers) to employ analytic reason- 
ing strategies in order to reject belief-disconfirming evi- 
dence (Lewandowsky & Oberauer, 2016). Nevertheless, 
it is important to emphasize that these findings apply 
only to those who hold extreme beliefs about publicly 
debated topics and cannot be taken as a complete rebut- 
tal of the abovementioned enlightenment heuristic. 
Carefully constructed and compelling counterargu- 
ments, such as the mechanistic explanation of global 
warming (Ranney & Clark, 2016), could lead to attitudi- 
nal change, as they are difficult to refute (Lewandowsky 
& Oberauer, 2016). 


5. Public Understanding of Science as a Play 
and a Stage for the Tension between Normative 
and Descriptive Accounts of Rationality 


The relationship between actual reasoning and nor- 
mative models of reasoning is a core issue within con- 
troversies about the nature of rationality (see also the 
Introduction by Knauff & Spohn, chapter 1.1 by Sturm, 
and chapter 1.2 by Evans, all in this handbook), even if 
science communication and citizens’ understanding of 
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science are mostly not explicitly discussed in controver- 
sies about the nature of rationality. The boundedness of 
nonexperts’ understanding of science points to a tension 
between the way in which science (as an ideal epistemic 
agent) reasons about the world and the way nonscien- 
tists do so in everyday contexts. In a sense, the public 
understanding of science is a case of this tension (merely 
a further play), just as reasoning within other domains of 
knowledge and of life are other plays. In another sense, 
the public understanding of science is also the overarch- 
ing societal context (the stage for all these plays) for 
rationality, because—as was argued at the beginning of 
this chapter—there is a strong nexus between rationality 
and science. If it is true that science provides the means 
for discerning between different beliefs about the world 
on a rational basis, then the public’s capability for ratio- 
nal reasoning is inevitably entrenched in its capability 
to understand science and its models of the world (i.e., 
scientific knowledge). When descriptive accounts of 
rationality aim to scrutinize the variance of rational rea- 
soning in the general public, they do so on the stage of 
public understanding of science. Therefore, assessments 
of this variance (Stanovich, 2016) include subscales mea- 
suring scientific beliefs about the world. 

Within the controversies about the nature of rational- 
ity, there have been several accounts arguing that the 
boundedness of reasoning (from a normative point of 
view) is no limitation for other kinds of rationality (see 
chapter 8.5 by Hertwig & Kozyreva and chapter 10.6 by 
Cosmides & Tooby, both in this handbook). Like others 
within research on the public understanding of science, 
we, too, argue in this vein. Judging the trustworthiness 
of researchers and of science in general is a rational strat- 
egy for overcoming the boundedness of a direct under- 
standing of science. The topic of this reasoning about 
trustworthiness is science as a social system, not science 
as a cognitive structure with its unlimited depth of pos- 
sible causal relations. 

Citizens’ capability to make such judgments is less 
bounded than their capability to judge which claims from a 
set of competing claims are true.* Science is a social subsys- 
tem of modern societies. Judgments about the trustworthi- 
ness of its actors require knowledge about this subsystem. 
Knowledge about the scientific consensus regarding a cer- 
tain topic belongs to this knowledge. Knowledge about 
science institutions, about their affiliation with different 
(vested) interests, about reasons for dissent among scien- 
tists (Thomm, Hentschke, & Bromme, 2015), and about 
limitations and failures of science experts (Burgman, 
2016) are further examples. The very fact that especially 
deniers of climate change underestimate the consensus 
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about this topic (van der Linden, Leiserowitz, Feinberg, & 
Maibach, 2015), even if they have some understanding of 
its mechanisms, points to the role of this kind of knowl- 
edge for trust judgments. Of course, such knowledge must 
be acquired, and its acquisition is related to a wide range 
of educational experiences.’ Nevertheless, this knowledge 
is more general than knowledge about specific science 
topics, because the criteria for judging the trustworthiness 
of different sources are less domain specific than the crite- 
ria for judging the validity of a specific knowledge claim. 
Furthermore, the social subsystem of science has common 
features with other subsystems within a society (educa- 
tion, health, industry) in which citizens have personal 
experience. Hence, knowledge about criteria for making 
rational judgments about the trustworthiness of science 
could be linked to these experiences. For example, a pos- 
sible detrimental effect of vested interests on a source’s 
epistemic trustworthiness could be experienced also in 
everyday contexts. Another example: consensus matters 
also for establishing what is true within other domains of 
rationality (Mercier & Sperber, 2017). This is not to argue 
that such reasoning is infallible and immune to the previ- 
ously described effects of motivated reasoning. However, 
reasoning about science as a trustworthy source of knowl- 
edge contributes to maintaining a broad stage for rational- 
ity within society. 
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Notes 


1. “Science” is used here in the comprehensive sense of the 
German notion Wissenschaft, including the social sciences as 
well as the humanities. 


2. Here we use the analytic concept “public understanding 
of science,” which embraces all kinds of citizens’ encounters 
with science, while the strategic notion of “Public Understand- 
ing of Science” typically refers to educational attempts (e.g., 
campaigns, books, events) to improve the public appreciation 
of science or to deliberate marketing attempts for increasing 
the public’s acceptance of science (Bromme & Goldman, 
2014). 


3. Peels (2020), in a critique of scientism, provides convincing 
arguments why it is not justifiable to claim that only scientific 
propositions could be deemed rational. Gottlieb and Lombrozo 
(2020) provide examples of, as well as reasons for, the limits of 
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scientific explanations as well as people’s intuitions about such 
limits. 


4. The concept of epistemic rationality is even more strongly 
linked with science; it refers to “how well beliefs map onto the 
actual structure of the world” (Stanovich, 2012, p. 434), and 
beliefs about this actual structure could not be as arbitrarily 
established as beliefs about aliens. 


5. The emphasis on the boundaries of the public’s under- 
standing of science should not be mistaken for the deficit 
model of science communication. This model assumes that 
it is just a lack of appropriate information that predicts 
people’s rejection of science. Recent reports on PUS start with 
a stark rejection of this model, motivating a focus on other 
predictors like values and attitudes (e.g., National Academies 
of Sciences, Engineering, and Medicine, 2017; but see Sul- 
dovsky, 2016). Such predictors will be discussed below, but 
we would argue that the interplay between attitudes, values, 
and knowledge within reasoning could not be understood if 
the very fact that there are boundaries of understanding is 
downplayed. 


6. Keren (2018) provides arguments (based on a social episte- 
mology approach) why reliance on the experts’ consensus is 
a rational way of reasoning for nonexperts, while it would be 
inappropriate for science as an epistemic community. 


7. Druckman and McGrath (2019) argue that all reasoning is 
motivated and that these cases of denying scientific evidence 
are not due to a deliberate rejection of evidence but to different 
criteria for what counts as evidence. 


8. Interestingly, representative democracy is based on a similar 
assumption. All citizens—as voters—are deemed to be able to 
make informed choices between different politicians (Goldman, 
1999; critically Baurmann & Brennan, 2009). It is assumed that 
they have sufficient capability to judge rationally whom to trust 
for solving political problems, even when they could not solve 
these problems by virtue of their own capacities. 


9. Recent educational approaches for analyzing and fostering 
citizens’ capacity to deal with information found in the Inter- 
net put a strong emphasis on such trust judgments (Barzilai & 
Chinn, 2019; Stadtler, Bromme, & Rouet, 2018; Tabak, 2015). 
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Henry Markovits 


Summary 


Understanding how rationality develops through child- 
hood to adulthood is important for both theoretical and 
practical reasons. However, existing theories provide 
very different and seemingly conflicting descriptions, 
with some postulating the existence of very early forms 
of biologically based rationality, while others describe a 
long developmental process leading to explicit principles 
guiding rational thought. In this chapter, these different 
approaches will be described. An initial distinction is 
made between explicit, metacognitive forms of rational 
thinking and the implicit structure of inferences. Within 
each, theories and data that describe the developmental 
course of reasoning are analyzed. While it is not possible 
to point to a clear synthesis, the continual interaction 
between empirical knowledge and logical reasoning is a 
consistent component in all of these approaches. 


1. Understanding the Development of Rationality 


Development is generally characterized by progression 
from simple to more complex forms of cognitive func- 
tioning, with or without the aid of biological under- 
pinnings. By analogy, one might expect children to 
become increasingly more rational. But this is far from 
universally supported. As with the adult literature, there 
is little consensus about the nature of rationality either 
in a broad or in a more restricted sense. The different 
developmental approaches that will be examined here, 
while seemingly divergent, indicate the complexity of 
the underlying developmental trajectory. As will be 
seen, the existence of irrational behavior in adults is 
coupled with evidence of forms of logical reasoning in 
very young children. Minimally, attempting to trace the 
development of rational thinking must require incorpo- 
rating different systems, which might well have very dif- 
ferent developmental paths. 

Understanding the development of rationality is fur- 
ther complicated by the difficulty of clearly defining 
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what rationality is in adults and even more in children. 
This is in fact the aim of many of the chapters in this 
handbook, and it would be impossible to repeat these 
arguments in a developmental context. However, most 
developmental studies that have looked at rationality 
specifically examine the ability of children to make infer- 
ences or judgments that correspond to standard logical 
norms, and for the purposes of this discussion, we will 
mostly use this general definition (for a broader perspec- 
tive, see chapter 2.4 by Fiedler, Prager,  McCaughey, this 
handbook). However, even with this relatively straight- 
forward definition, there is little consensus as to how, 
or even whether, the ability to make rational judgments 
develops. This inconsistency is probably most clearly 
shown by the existence both of theories that project 
clear developmental increases in children’s rationality 
and of theories that suppose that even adults are not 
rational. In addition, developmental studies have exam- 
ined very different phenomena, which are often difficult 
to compare. However, we can make one useful distinc- 
tion that reflects different ways of examining this ques- 
tion. Specifically, we can distinguish between explicit 
and implicit rationality. The former refers to the study of 
the reasons that people give either to justify their own 
inferences or to explicitly account for differences in the 
level of adequacy of different forms of inference. The lat- 
ter refers to the logical adequacy of judgments or infer- 
ences that people actually make. 


2. The Development of Explicit Rationality 


The study of the development of explicit rationality has 
two major poles, metalogical and argumentation. 


2.1 Metalogical Development 

“Metalogic” refers to the ability to think about one’s 
own reasoning processes and can be seen as a subset of 
more general metacognitive processes (Flavell, 1979). 
The most direct theory describing the development of 
reasoning abilities is Moshman’s account of reasoning 
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(Moshman, 1990, 2004, 2013; Moshman & Franks, 1986). 
Moshman makes a distinction between the implicit pro- 
cesses that are used to make specific inferences (which 
will be discussed later) and rationality, which can be 
defined as the ability to explicitly represent higher-level 
metalogical components of reasoning, such as concepts 
of inferential validity and logical necessity (Christo- 
forides, Spanoudis, & Demetriou, 2016; Demetriou, 
Spanoudis, & Shayer, 2014; Moshman, 2011; Moshman 
& Franks, 1986). Metalogical understanding is the result 
of self-reflective processes that allow the abstraction of 
general principles from more specific forms of inference. 
In other words, rationality is developed through think- 
ing about implicit inferences, by the reasoner trying to 
understand why these inferences are good or bad. The 
process is conceived of as a continuous one, leading to 
complex forms of epistemological understanding whose 
development goes well through adulthood (Hallett, 
Chandler, & Krettenauer, 2002) but with no necessary 
endpoint. More specifically, Moshman (2014) defines 
reasoning as involving inferential processes within the 
more general constraints of a metacognitive understand- 
ing of the goals and aims that animate a reasoner. Such 
forms of reasoning are inherently contextual, with the 
criteria defining “good” reasoning varying according 
to the domain in question. In other words, inferential 
processes are cognitive tools in the service of broader 
metacognitive goals and understanding, that is, ratio- 
nality comprises the broader principles that guide spe- 
cific inferences. Both the cited empirical studies and 
theoretical underpinnings suggest that with this defini- 
tion, rationality undergoes an ongoing developmental 
process, such that the ability to be rational increases 
with experience and age. The clear emphasis on reason- 
ing as inferences under metacognitive control thus sug- 
gests that rationality is late-developing, since it requires 
fairly complex forms of metacognitive understanding. 
Evidence of early forms of logical inference are simply 
building blocks in a complex developmental progres- 
sion. For example, young children can consistently go 
from premises of the form “If P then Q; P is true” to cor- 
rectly infer “Q is true” but do not necessarily understand 
why this inference is necessary. The basic metalogical 
understanding of the notion of logical necessity (i.e., to 
explicitly distinguish between empirical truth and valid- 
ity) undergoes a clear developmental increase and is not 
typically shown before 10 to 12 years of age (Moshman 
& Franks, 1986). 

There is one important question that is left unanswered 
by this theory, which is the relationship between ratio- 
nality and inferential reasoning. Moshman’s approach 
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suggests that inferential processes are guided by meta- 
cognitive analyses. Indeed, one would expect a stronger 
relationship, so that the increased metacognitive aware- 
ness of the characteristics of reasoning that underpins the 
development of rationality should have a clear effect on 
the nature of the inferential processes that are used. There 
are almost no empirical data in a developmental context 
that addresses this question. It is worth citing recent 
work by Thompson (Newman, Gibb, & Thompson, 2017; 
Thompson, Prowse Turner, & Pennycook, 2011) that has 
examined the relationship between explicit degree of 
confidence in the rightness of judgments, which is one 
evaluation of metacognitive processes, and their logi- 
cality in adults. In fact, this work has found very little 
relationship between the two and certainly suggests that 
explicit metacognitive judgments do not necessarily cor- 
relate with the extent to which actual reasoning is logical 
(although see Markovits, Thompson, & Brisson, 2015). 
Thus, although it is very clear that the ability to explicitly 
analyze reasoning in terms of logical components devel- 
ops in a consistent manner, it is less clear that rationality 
defined in this manner has a specific impact on actual 
judgments. 


2.2 Argumentation 

Another important form of explicit understanding that 
shows clear developmental patterns is derived from the 
notion of argumentation (see also chapter 5.5 by Hahn & 
Collins, this handbook). “Argumentation” refers to the 
way that specific inferences and accompanying justifica- 
tions can be marshaled in order to either support a point 
of view or dispute an alternative viewpoint (Perelman, 
1971). Ideally, argumentation skills allow a conscious 
weighting of the relative logical strength of different 
forms of inference, which can be seen as a building block 
for rationality. More important, argumentation requires 
an interaction between people having different points of 
view. Although it is possible to conceive of argumenta- 
tion as an essentially one-way process (Mercier & Sper- 
ber, 2011), or a Vygotskian interaction between an adult 
and a child (Berk & Winsler, 1995), a stronger develop- 
mental viewpoint would suggest that the continual con- 
frontation between opposing forms of inference would 
generate some form of metacognitive resolution of the 
resulting conflict. In fact, this form of interaction places 
the role of argumentation within the general context 
of social constructivism (Doise & Mugny, 1981), which 
suggests that the social confrontation of different points 
of view may be a critical component of cognitive devel- 
opment in general. There are in fact several studies that 
have shown the beneficial effect of interactions between 
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children of different cognitive levels on basic Piagetian 
concepts (Ames & Murray, 1982; Murray, 1972). Such 
interactions have been shown to improve individual 
levels of logical reasoning (Moshman & Geil, 1998). 
More specifically, Kuhn and colleagues have shown that 
argumentation skills have a clear developmental trajec- 
tory (Kuhn, 1991, 1993; Kuhn & Udell, 2003). Promot- 
ing argumentation within the context of a collaborative 
social context produces a clear improvement in the abil- 
ity of individuals to use argument strategies of increasing 
sophistication (Crowell & Kuhn, 2014; Kuhn & Crowell, 
2011; Kuhn, Zillmer, Crowell, & Zavala, 2013). In other 
words, this general approach considers that rationality is 
a natural end state of a process by which different forms 
of arguments are directly confronted. This in turn sup- 
poses, consistent with the general tenets of social con- 
structivism, that such a confrontation allows explicit 
acknowledgment of arguments that are logically supe- 
rior and promotes metacognitive development leading 
to a more explicit understanding of what makes argu- 
ments logically superior. 

As with the metalogical conception of rationality, 
this conception of argumentation sees it as a develop- 
mental process that acts on explicit representations, 
one that produces a consistent evolution toward greater 
rationality. However, it is also worth noting that a recent 
approach to argumentation proposed by Mercier and 
Sperber (2011) suggests a more unilateral conception 
with a variable relationship to rationality. They suggest 
that argumentation has evolutionary underpinnings for 
which the main aim is to achieve a form of dominance, 
by convincing others of the strength of an individual’s 
point of view. In fact, this approach suggests that logical 
reasoning has evolved in order to support argumenta- 
tion and that logic and rationality are rhetorical devices 
intended to construct an individual point of view strong 
enough to overcome other arguments (Mercier, 201 1a, 
2011b). Within this general perspective, rationality is 
seen as a skill that can be deployed or not, depending 
on the specific aims and context. There is some evidence 
that supports this point of view. For example, Klaczynski 
(2001) asked adolescents to evaluate arguments in terms 
of their logical adequacy. Older adolescents were quite 
good at picking out logical inconsistencies in relatively 
neutral arguments. However, where the same incon- 
sistencies were used to support conclusions for which 
these adolescents had emotional beliefs, they tended 
very strongly to consider these arguments to be logically 
appropriate. 

Both metalogical and argumentative approaches con- 
sider that there is a clear developmental progression in 
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the ability of children and adolescents to explicitly con- 
sider the logical status of inferences and arguments. Both 
see this progression to be long and relatively difficult, 
suggesting that rationality requires a long developmen- 
tal sequence. One major difference between these two 
points of view is the locus of this development. Mosh- 
man’s approach suggests that metacognitive control and 
understanding is essentially an individual process, medi- 
ated by thinking about the structure of inferences. The 
notion of argumentation, by contrast, suggests that the 
development of an explicit rationality is the product of 
an ongoing process of social interaction. However, in 
both cases, there are reasons to believe that the kinds of 
inferences and judgments that people make are not fully 
accounted for by their explicit understanding. 

In fact, there is a great deal of evidence that the infer- 
ences people actually make are highly variable. Both of 
the approaches that we have examined clearly suggest 
that there is an ongoing developmental progression in 
the abilities of people to consciously understand the 
basic principles underlying explicit rationality. If this 
was accompanied by a corresponding control of the 
actual inferences and judgments that people make, then 
we would expect a clear developmental progression 
in the nature of the inferences that were made, with 
adults showing a clearly consistent pattern of rational 
judgments. However, there is a great deal of empiri- 
cal evidence that adults are much less rational in their 
judgments than one would expect from such a point of 
view. Both the biases-and-heuristics program of Tversky 
and Kahneman (2004) and more current dual-process 
theories (Evans & Stanovich, 2013; Sloman, 1996) have 
clearly shown that people’s judgments and inferences 
are far from logically appropriate. In fact, much of the 
research on the development of reasoning has examined 
what kinds of inferences people make irrespective of 
their explicit reasons, with the idea that the structure of 
these inferences gives an implicit picture of the under- 
lying level of rationality. 


3. The Development of Implicit Rationality 


Before examining the different developmental theories 
that have attempted to explain how inferential reason- 
ing develops in children and adolescents, it is useful to 
present the general interplay between logic and knowl- 
edge, which has been one of the key debates in this field. 
As mentioned previously, research on reasoning in adults 
has shown a remarkable level of what can be construed 
as irrationality. Although there are many factors under- 
lying these effects, one of the more important of these 
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is the strong influence of people’s knowledge and beliefs 
on the inferences that they make. This has led some cur- 
rent theorists to suggest that classical “logic” may not be 
the best model to understand the adequacy of people’s 
reasoning but that some form of Bayesian probability 
calculus might be better adapted to the reality of human 
reasoning (e.g., Evans, Over, & Handley, 2005; Oaksford 
& Chater, 2007; see also chapter 4.5 by Chater & Oaks- 
ford, this handbook). In other words, rationality might 
be construed as the ability to reason in a way that is 
consistent with one’s knowledge. Developmental theo- 
ries have, in contrast, tended to examine changes in the 
ability to generate inferences that correspond to logical 
norms, in a way that is relatively independent of empiri- 
cal knowledge. Nonetheless, the interplay between logic 
and knowledge is critical to understanding how reason- 
ing develops, and this is one of the major themes of the 
developmental theories that will be described. 

There are, however, different mechanisms that could 
explain interactions between contextual knowledge and 
reasoning. On the one hand, it has been claimed that 
biology can account for at least some of the ways that 
people reason, since evolutionary mechanisms could 
create ways of thinking that are rational in the sense 
that they allow inferences and judgments that directly 
reflect important characteristics of evolved performance 
and social structures. Other explanations involve the 
development of specific cognitive processes that follow 
a developmental trajectory and results in developmen- 
tal changes in reasoning abilities. We will examine dif- 
ferent approaches to development of reasoning within 
this general distinction. We start by examining Piaget’s 
theory, since this combines both biological and devel- 
opmental perspectives, and it was the impetus for many 
subsequent developmental theories. 


3.1 Piaget’s Theory 

Piaget’s theory of cognitive development proposes that 
development occurs through the constant interaction 
between cognitive assimilation and accommodation, 
which are biological processes the use of which leads 
to a slow but progressive increase in the power of rea- 
soning processes (Piaget, 1972). Without going into 
too much detail, one of the key factors characterizing 
the increasing ability to make logical inferences is the 
existence of three major levels of abstraction, with 
the transition being mediated by a process referred to 
as reflexive abstraction (Piaget, 2001). This is the abil- 
ity to think about lower-level reasoning and represent 
the commonalities in more abstract form. The initial 
sensorimotor level uses action and perception-based 
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schemas. A second level, concrete operations, uses an 
initial form of abstraction that is linked to the general 
characteristics of physical objects. The third level, for- 
mal operations, uses a higher level of abstraction that 
allows the manipulation of general concepts that are 
no longer tied to physical objects. Progression through 
each of these three general levels leads to an increasing 
ability to make inferences and judgments that, at least 
in theory, reflect the logical structure appropriate to a 
given level. The key to understanding this progression 
is the idea that at any given level, children are “locally 
rational.” That is, they make inferences that are con- 
sistent with the structural information available at any 
given level. Thus, sensorimotor inferences are consis- 
tent with the structure of physical action schemas, and 
so on. It is not until the formal operational level that 
inferences become rational in a larger sense, since the 
structure of formal operational schemas functions at a 
very high level of abstraction. Interestingly, empirical 
examination of this model has led to two very different 
criticisms. On the one hand, there is research that sug- 
gests that even very young children are capable of being 
logical in a way that should only be accessible to formal 
operational adolescents and adults (Hawkins, Pea, Glick, 
& Scribner, 1984). On the other hand, as we have already 
seen, much research suggests that even educated adults 
are not consistently logical. 

One theory that has attempted to account for the latter 
results within the Piagetian framework is Overton’s com- 
petence performance model (Overton & Ricco, 2011). 
This theory makes a distinction between the theoretical 
competence to reason logically, which follows the basic 
trajectory described by Piaget, and use of this compe- 
tence, which is subject to a variety of performance-related 
factors. Consistent with this approach, several studies 
have found that manipulations designed to improve rea- 
soning have effects that are clearly age related (O’Brien & 
Overton, 1980, 1982). 

However, other approaches have focused on empiri- 
cal results that indicate that very young children can 
sometimes reason logically. These have sometimes been 
interpreted within a biological perspective. 


3.2 Evolution and Reasoning 

Natural-logic theory A very different approach is pro- 
vided by Braine’s natural-logic theory (Braine, 1978; see 
also chapter 3.2 by O’Brien, this handbook). This theory 
has an empirical basis in studies that show that under 
certain circumstances, and at least with a limited range 
of inferences, even quite young children are able to make 
logical deductions (e.g., Hawkins et al., 1984). There are 
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two major components to the theory. The first is the 
idea that people have specific algorithmic inference 
rules embedded into the cognitive system. These rules 
are essentially syntactic in form, and they allow analysis 
of the syntax of a given inference in order to generate a 
conclusion that is theoretically independent of context. 
The second component is a biological underpinning. 
The basic argument is that in certain circumstances, it 
is absolutely critical for individual survival to be able to 
make very rapid and accurate inferences. Thus, biology 
equips people with selected inference rules that allow 
consistent production of logical inferences in situations 
where these inferences are critical. One such example is 
the modus ponens inference, which is a basic inference 
from conditional premises of the form “If P then Q; P is 
true” to “Q is true.” 

Although this model of reasoning suggests that people 
are equipped with rules of inference that always give 
the logically correct response, existing empirical data 
show much more variability than would be suggested 
by the model. In order to account for such variability, 
the theory adds an interpretational module to the basic 
mechanisms underlying reasoning. This in turn creates 
an interaction between empirical knowledge, especially 
knowledge of the pragmatics of a given situation, and 
reasoning. 


Cheater detection modules Another strong biologi- 
cal approach to reasoning was suggested by Cosmides 
(1989; see also chapter 10.6 by Cosmides & Tooby, this 
handbook). This theory had its origins in studies of the 
Wason selection task (Wason, 1968), which was a task 
first conceived as a way of showing that even very well- 
educated adults did not always make formally correct 
inferences. People were shown four cards, each of which 
had a number on one side and a letter on the other. A 
visible top of each card showed, respectively, a vowel, a 
consonant, an even number, and an odd number. Sub- 
jects were presented with a conditional rule of the form, 
“Tf a card has a vowel on one side, then it has an even 
number on the other.” They were asked to choose which 
cards they would turn over in order to either confirm or 
disconfirm the rule. The logically correct answer is, “The 
card with a vowel and the card with an odd number.” 
Both the original study, which was done with philoso- 
phy students, and several replications consistently found 
remarkably low levels of correct responding. Subsequent 
studies examined conditions under which the rate of 
correct responses was higher. One of the more intriguing 
results was that use of certain kinds of social rules led to 
very high levels of correct responding, for example, “Ifa 
person wants to drink at a bar, then they need to be over 
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21” (Cheng & Holyoak, 1985). With such a rule, people 
have no problems choosing cards that represent a per- 
son drinking and a person who is less than 21. In order 
to explain these results, Cosmides proposed that these 
social rules lead to good logical performance because 
they tap into a specific cheater detection module that 
has biological origins. The general proposition was based 
on the idea that social exchange and explicit or implicit 
contracts are a vital part of human social structure and 
have been so for a very long time. However, cheating is 
also a basic part of social interactions. Thus, it is argued 
that, in order to maintain a social structure based on 
exchange, people have developed a specific, special- 
ized way of identifying cheaters. Consistent with this 
point of view, studies have shown that even very young 
children are able to produce logically correct inferences 
on some forms of the Wason selection task (Cummins, 
1996; Girotto, Light, & Colbourn, 1988). Critically, such 
a module is biologically designed to allow people to 
interact in social situations in a way that maximizes the 
probability of successfully concluding a social contract 
but is not designed to reason logically. The fact that use 
of this module leads to the logically correct response in 
specific situations is thus basically incidental. In other 
words, this approach suggests that biology may have 
equipped people with inference mechanisms that are 
what we might call contextually rational. This means 
a kind of reasoning that accurately reflects the specific 
dimensions of a given social situation, which can be 
seen as rational, in the sense that it accurately reflects 
the specific parameters of the situation in question, but 
is not necessarily logical in a more formal sense. 


Biologically based social reasoning It is interesting 
to examine some possible extensions of this notion 
of social reasoning, which shows the extent to which 
implicit inferences are affected by social schemas. One 
good example of this mechanism is given by attachment 
theory (Bowlby, 1969). The attachment system is biolog- 
ically based and is designed to ensure the proximity of 
offspring to their principal caregiver during the period 
in which the offspring are most physically vulnerable. 
By itself, the system is a behavioral one that functions 
through an intricate mechanism of signals and internal 
emotions (Ainsworth, Blehar, Waters, & Wall, 1978). 
Depending on the reactions of the caregiver in situa- 
tions of stress and/or anxiety, children create differing 
expectations, which results in very different forms of 
attachment behavior. For example, if a child’s caregiver 
is physically affectionate and comforting in situations 
of stress, they will have a secure attachment. If the care- 
giver is cold and distant, this will result in an avoidant 
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form of attachment. Critically, this behavioral cycle is 
internalized in the form of an internal model of a child’s 
attachment figure (Bretherton & Munholland, 2008). 
This internal model is then used to interpret the behavior 
of potential romantic partners (Hazan & Shaver, 1987). 
In other words, the internal model generates inferences 
about how important people in a person’s life will react, 
which in turn reflects their early life experiences (Baldwin, 
1992). 

As we can see from these different approaches, biol- 
ogy has been claimed to underlie both a universal sys- 
tem of strictly logical deduction and specialized systems 
of contextual inferences that accurately mirror the con- 
straints of specific social situations. These theories are 
designed to explain empirical data that show that cer- 
tain types of inferences (which can be considered ratio- 
nal in the sense that they correspond to either logical or 
social norms) are consistently made by even very young 
children. However, there is also a very strong body of 
empirical evidence that shows important developmental 
changes in children’s and adolescent reasoning. What 
we might call information-processing theories attempt 
to specify the underlying mechanisms that could explain 
these developmental patterns. 


3.3 Information-Processing Approaches 

to the Development of Reasoning 

As we have previously indicated, these approaches can 
be seen as the result of two tendencies. On the one hand, 
there is clear evidence that the efficiency of cognitive 
processing increases over time. On the other hand, it is 
also the case that people’s knowledge about the empiri- 
cal world increases concurrently. Both of these factors 
are conceivably linked to the development of reasoning 
abilities. The following theories put different weights on 
these factors. 


Barrouillet’s mental model theory One of the key factors 
to understanding the efficiency of cognitive processes 
is the capacity of working memory. This is a critical 
factor in the developmental approach to reasoning pro- 
posed by Barrouillet and colleagues (Barrouillet, Gros- 
set, & Lecas, 2000; Barrouillet & Lecas, 1999; Gauffroy 
& Barrouillet, 2011). Their theory is based on mental 
model theory (Johnson-Laird & Byrne, 2002, which is 
a semantics-based approach to understanding reason- 
ing (see chapter 2.3 by Johnson-Laird, this handbook). 
This attempts to model the processes that are used when 
people make deductions on the basis of a given set of 
major and minor premises. Since one of the principal 
forms of reasoning that have been examined by this 
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theory, and other developmental theories, is conditional 
(if-then) reasoning, we will use this as an example. 

Conditional reasoning examines the inferences that 
are made on the basis of a major premise of the form 
“Tf P then Q.” There are four inference forms that char- 
acterize such reasoning, constructed by minor premises 
having true or false forms of the antecedent (P) or conse- 
quent (Q) terms. Two of these lead to logically necessary 
conclusions. These are modus ponens (MP), “If P then Q; 
P is true,” which leads to the necessary conclusion “Q is 
true,” and modus tollens (MT), “If P then Q; Q is false,” 
which leads to the necessary conclusion “P is false.” The 
other two forms have what can be referred to as invited 
conclusions, which are however not logically necessary. 
These are affirmation of the consequent (AC), “If P then 
Q; Qis true,” which leads to the invited conclusion “P is 
true,” and denial of the antecedent (DA), “If P then Q; 
P is false,” which leads to the invited conclusion “Q is 
false.” However, for both the AC and the DA inferences, 
the logically correct response is to deny the invited 
conclusions. 

Mental model theory proposes that people have a 
theoretical representation of conditionals that include 
combinations of antecedent and consequent terms that 
correspond to the true rows of the logical truth-table 
representation. However, the actual process people use 
is complicated by the fact that constructing such inter- 
nal representations requires cognitive resources. Thus, a 
two-stage process is proposed that allows maximal use 
of memory in particular. When reasoning with a con- 
ditional statement “If P then Q,” people start with an 
initial representation (model) of the statement, which 
simply represents the conjunction of the antecedent and 
the consequent terms, in the following way: 


PQ 


for each of the four different inferences, then the model 
is scanned for cases involving the minor premise, par- 
ticularly for cases in which the minor premise is associ- 
ated with two different possible states. With this initial 
representation, people will make the MP inference and 
will accept the invited conclusion to the AC inference. 
This initial representation can in some circumstances 
be fleshed out by the addition of other models that are 
also consistent with the conditional relation. The sim- 
plest of these leads to the following representation: 


P Q 
not-P not-Q 


These models correspond to a biconditional interpreta- 
tion, that is, “P is true if and only if Q is true.” With this 
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representation, people will make the correct inferences 
for the MP and the MT forms, and they will accept the 
invited conclusions for the AC and the DA forms. Such 
an interpretation corresponds to a pattern of inferences 
very frequently observed in children. 

Finally, the full interpretation of conditionals requires 
the addition of a further model, leading to the following 
representation: 


P Q 
not-P Q 
not-P not-Q 


With such a representation, people will make the cor- 
rect inferences for the MP and the MT forms, and they 
will reject the invited conclusions for the AC and the 
DA forms. Mental model theory supposes that the inter- 
pretation given to conditional statements is a basically 
semantic process based on understanding of language, 
modulated by pragmatic considerations (Johnson-Laird 
& Byrne, 2001). In other words, people will generate 
models of full conditionals unless cognitive and/or con- 
textual constraints limit their ability to do so. 

Barrouillet’s theory makes the very strong claim that 
working-memory constraints are the primary develop- 
mental factor determining how children and adults rea- 
son (Barrouillet & Lecas, 1999). The key explanatory factor 
is the idea that working-memory capacity constrains the 
number of models that people can actively manipulate. 
Young children have very limited working-memory capac- 
ity and are limited to a single model. This leads to a con- 
junctive interpretation of conditionals. Older children 
have increasing working-memory abilities and are able to 
manipulate two models, which leads to the generation of 
a biconditional interpretation. Finally, much older adoles- 
cents and adults have sufficient working-memory capacity 
to generate the three models required for the full interpre- 
tation of conditionals. 

Empirical evidence for this model has used tasks that 
attempt to directly examine the interpretation that is 
made of conditional statements. One such task is the 
truth-table task, in which people are given conditional 
statements and are asked whether each of the four com- 
binations composed of true or false antecedent and 
consequent terms makes the conditional true, false, or 
neither true nor false. Several studies have found a clear 
developmental progression from a conjunctive interpre- 
tation, which is predominant until 8 or 9 years of age, 
to a biconditional interpretation, predominant until 11 
to 12 years of age, to a conditional interpretation, which 
appears in mid-adolescence, with an additional relation- 
ship to measures of working memory as hypothesized 
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(Barrouillet & Lecas, 1999; although see Markovits, Bris- 
son, & de Chantal, 2016). 

This theory suggests that the ability to make logical 
inferences resides primarily in the increasing capacity of 
the cognitive system. Interestingly, it supposes that the 
underlying logic is determined by the way that state- 
ments are interpreted and that such interpretations are 
theoretically accessible even to young children. It should 
be noted that this theory does acknowledge the existence 
of interpretations reflecting contextual, pragmatic factors, 
which lead to inferences that are not logically adequate 
but pragmatically appropriate (Barrouillet, 2011). None- 
theless, it does incorporate one of the more controversial 
aspects of mental model theory, which is the idea that in 
ideal circumstances, reasoning will conform to the norms 
of standard logic. Thus, rationality in a larger sense can 
be seen as the result of a more adequate cognitive appara- 
tus that can increasingly profit from the basic capacity for 
logical reasoning. 


Semantic redescription theory Another approach that 
is based on mental model theory has been developed 
by Markovits and colleagues (Markovits, 2004; Marko- 
vits & Barrouillet, 2002). This theory postulates that the 
basic interpretation of conditionals is used as a retrieval 
cue for the activation of information present in seman- 
tic memory, which is then incorporated into a mental 
model. Two major classes of information have been 
shown to impact the way that people reason. The first of 
these are alternative antecedents. These are propositions 
A that differ from P and for which “If A then Q” is true. 
For example, consider the conditional rule “If a rock is 
thrown at a window, then the window will break,” for 
which examples of alternative antecedents are “if a chair 
is thrown at a window,” “if any hard object is thrown at 
a window,” and so on. A great many empirical studies 
have shown that the tendency to accept or to reject the 
invited conclusions to the two invalid inferences (AC, 
DA) is directly related to the accessibility of alternative 
antecedents in memory (Cummins, 1995; Cummins, 
Lubart, Alksnis, & Rist, 1991; Markovits, Fleury, Quinn, 
& Venet, 1998; Markovits & Vachon, 1990; Thompson, 
1995). Another form of information that is cued by con- 
ditional rules are disabling conditions (Cummins et al., 
1991). These are conditions that allow P to be true with- 
out Q being true. For example, a disabling condition for 
the conditional rule presented above would be “The win- 
dow is made of Plexiglas.” Studies have shown that the 
tendency to accept the logical conclusions to the MP 
and MT inferences is related to the relative accessibil- 
ity of disabling conditions in memory (Cummins, 1995; 
Cummins et al., 1991; De Neys, Schaeken, & d’Ydewalle, 
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2003). One of the first implications of this theory is 
that the ability to reason logically is associated with the 
efficiency of people’s retrieval processes (Markovits & 
Quinn, 2002). However, the retrieval processes used dur- 
ing reasoning are not targeted but tend to result from 
activation of all the information that is associated with 
the premises (Janveau-Brennan & Markovits, 1999; Mar- 
kovits & Potvin, 2001). Thus, the tendency to reject the 
AC and DA inferences is also associated with the ten- 
dency to reject the MP and MT inferences. This in turn 
suggests the importance of processes that are used to 
inhibit the retrieval of information that is inconsistent 
with the basic premises (Handley, Capon, Beveridge, 
Dennis, & Evans, 2004; Simoneau & Markovits, 2003). 
On a general level, this theory suggests that the ability to 
reason logically depends on the amount of information 
stored in semantic memory, the ability to retrieve this 
information, and the ability to inhibit retrieval of inap- 
propriate information (i.e., information that contradicts 
the major premise). Since all of these factors increase 
developmentally, this results in a general improvement 
in the ability of children and adolescents to reason logi- 
cally (Janveau-Brennan & Markovits, 1999). 

However, this theory also postulates the existence of 
a redescriptive process as suggested by Karmiloff-Smith 
(1995), which produces descriptions of alternative ante- 
cedents of an increasingly abstract nature, resulting in a 
developmental increase in the ability to be logical with 
premises that are more abstract. Empirical studies that 
have examined the ability of children and adolescents to 
make consistently logical inferences have shown a clear 
age-related progression related to the content of the con- 
ditional rules, which in turn is related to the extent to 
which these rules allow accessibility of alternative ante- 
cedents and/or disabling conditions. These can be sum- 
marized by the following sequence: 


* category-based conditional rules (if an animal is a 
dog, then it has legs), 


* causal conditional rules (if a rock is thrown at a win- 
dow, then the window will break), 


* contrary-to-fact conditional rules (if a feather is 
thrown at a window, then the window will break), 
and 


* abstract conditional rules (if XY, then it will blrp). 


Studies have shown that children as young as seven 
or eight years of age are able to reason logically with 
category-based conditionals (Markovits, 2000; Markovits 
& Thompson, 2008). However, reasoning logically with 
causal conditionals is more difficult; it is not observed 
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before 11 or 12 years of age (Janveau-Brennan & Marko- 
vits, 1999; Markovits, 2017). Reasoning with contrary-to- 
fact conditionals and with abstract conditionals is much 
more complex, with the former showing high rates of 
logical reasoning only at late adolescence (Markovits, 
2014; Markovits & Vachon, 1989), while abstract rea- 
soning is not found at a high rate even with educated 
adults (Markovits & Lortie-Forgues, 2011; Markovits & 
Vachon, 1990). The redescriptive theory assumes that 
people think about the kinds of reasoning that they can 
do well and can use this to construct a more abstract 
representation of the processes involved, which then 
allows a higher level of reasoning. 

This further supposes that the four categories of prem- 
ise described previously correspond to two major levels 
of abstraction. The first two, comprising reasoning with 
category-based and causal premises, require retrieval 
of information that is directly or indirectly available 
in semantic memory. The second two, reasoning with 
contrary-to-fact and abstract premises, require the active 
construction of relevant information. Evidence for this 
comes from studies that have shown that it is possible to 
improve reasoning by asking reasoners to simply gener- 
ate explicit alternatives for premises that are at a higher 
level of abstraction than those used for reasoning (Mar- 
kovits, 2014; Markovits & Lortie-Forgues, 2011), while 
generating alternatives for highly familiar premises has 
no effect. In addition, these studies have found clear sup- 
pression effects showing that levels of logical reasoning 
with concrete and familiar premises are lowered when 
reasoners are first asked to reason with abstract premises 
(Markovits & Vachon, 1990). 


3.4 Divergent Thinking and Reasoning 

This theory also suggests an interesting relationship 
between logical reasoning and creativity. One of the 
more important factors that allow people to make logi- 
cal inferences based on conditional rules is the ability to 
maintain the conditional in memory and at the same 
time to activate alternative antecedents. This process 
is made more difficult when premises contain terms 
that are already strongly associated (Quinn & Marko- 
vits, 1998), so that overall rates of logical reasoning 
are related to the ability of reasoners to retrieve infor- 
mation that is outside the usual range suggested by the 
premises (Markovits et al., 1998; Markovits & Quinn, 
2002). This ability is very similar to that involved in 
divergent thinking, which is a key component of cre- 
ativity (McCrae, 1987). There is in fact some evidence 
that differences in divergent thinking drive the very 
early ability of children to reason logically (de Chantal, 
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Gagnon-St-Pierre, & Markovits, 2020; de Chantal & Mar- 
kovits, 2017; Markovits & Brunet, 2012). For example, 
very young children have great difficulty in accepting 
the uncertainty of even a very simple category-based 
inference, such as “If an animal is a dog, then it has four 
legs. An animal has four legs. Is it a dog?” However, if 
they are asked to do a divergent-thinking task (with- 
out examples) such as “We can make noise with many 
things. I’d like to know some ways of making noise,” 
their ability to accept uncertainty increases greatly. The 
general idea that rationality in a broader sense depends 
upon the growing ability of reasoners to envisage pos- 
sibilities (which is important for divergent thinking) 
and then to account for these possibilities when making 
inferences, is very similar to the interplay between pos- 
sibility and necessity that was identified by Piaget in his 
later works (Gauffroy & Barrouillet, 2011; Piaget, 1987). 


4. Conclusion 


While there is no real consensus about the way that 
rationality develops, the variety of approaches does sug- 
gest a multidimensional trajectory that might usefully 
be extrapolated from these theories and the associated 
empirical data. One of the themes is the interaction 
between context-dependent forms of reasoning and 
more general forms of reasoning, of which the stron- 
gest version is metacognitive and abstract. The simplest 
version of this would suggest that the earliest form of 
rational thinking is contextual, either due to biologi- 
cal underpinnings or early experience, with rational- 
ity implying reasoning that reflects specific situational 
constraints. Subsequent development would involve a 
gradual replacement of this form of reasoning with more 
abstract and general forms. 

However, one interpretation of the many studies 
that have examined the development of reasoning is 
that inferences and judgments can be seen as interac- 
tions between some form of rationality and a growing 
understanding of the structure of the empirical world, 
with the latter anchored in a biological substrate. Actual 
inferences can then be seen as the product of an ongo- 
ing exchange between children’s logical abilities and 
their increasing understanding of the empirical world. 
Each of these components is subject to developmental 
variation underpinned by clear increases in the compu- 
tational power of the cognitive system. Thus, develop- 
ment allows children to become both more logical and 
more conversant with the way that the empirical and 
social worlds are constructed. The paradox of develop- 
ment is that real-world knowledge is necessary for the 
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construction of an increasingly powerful ability to make 
rational judgments, and it is at the same time a driver 
of judgments that reflect the often irrational structure 
of the social and possibly the physical world (Artman, 
Cahan, & Avni-Babad, 2006). Accounting for this com- 
plex interaction is one of the major challenges of under- 
standing the development of rational thought. 
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15.2 Rationality and Intelligence 


Keith E. Stanovich, Maggie E. Toplak, and Richard F. West 


Summary 


There are individual differences in rational thinking 
that are less than perfectly correlated with individual 
differences in intelligence because intelligence and 
rationality occupy different conceptual locations in 
models of cognition. A tripartite extension of currently 
popular dual-process theories is presented in this chap- 
ter that illustrates how intelligence and rationality are 
theoretically separate concepts. The chapter concludes 
by showing how this tripartite model of mind, taken 
in the context of studies of individual differences, can 
help to resolve the Great Rationality Debate in cogni- 
tive science—the debate about how much irrationality 
to attribute to human cognition. 


1. What Intelligence Is—and Why It Is Not the Same 
as Rationality 


Because both intelligence and rationality have a plethora 
of definitions that differ across the various disciplines, it 
is not surprising that our chapter will begin with some 
definitional clarifications. We take our definitions from 
cognitive science, so those defaulting to other fields and 
disciplines may be confused if they insist on stipulating 
definitions we are not using. Also, if rationality and intel- 
ligence are to be compared and contrasted, each must be 
defined at a similar grain size, which we do in this chap- 
ter. That is, rationality defined in some ways would be 
a category error if directly compared with intelligence. 
Because intelligence is an individual difference concept 
to a psychologist, rationality must be too. That rules out 
certain definitions that are popular in philosophy and 
in lay discourse. 

We get surprised when someone whom we consider to 
be smart acts stupidly. When someone we consider to be 
not so smart acts stupidly, we tend not to be so surprised. 
But why should we be so surprised in the first case? A 
typical dictionary definition of the adjectival form of the 
word “smart” is “characterized by sharp quick thought; 


bright” or “having or showing quick intelligence or 
ready mental capacity.” Thus, being smart seems a lot 
like being intelligent, according to the dictionary. Dic- 
tionaries also tell us that a stupid person is “slow to 
learn or understand; lacking or marked by lack of intel- 
ligence.” Thus, if a smart person is intelligent and “stu- 
pid” means a lack of intelligence, then the “smart person 
being stupid” phrase seems to make no sense. 

However, a secondary definition of the word “stu- 
pid” is “tending to make poor decisions or careless 
mistakes”—a phrase that attenuates the sense of con- 
tradiction. Thus, the phrase “smart but acting dumb’— 
intelligent people taking injudicious actions or holding 
unjustified beliefs—means that folk psychology is picking 
out two different traits: mental brightness (intelligence) 
and making judicious decisions (rational thinking). If we 
were clear about the fact that the two traits are different, 
the sense of paradox or surprise at the “smart but acting 
foolish” phenomenon would vanish. What perpetuates 
the surprise is that we tend to think of the two traits 
as one, or at least that they should be strongly associ- 
ated. The confusion is fostered because psychology has 
a measurement device for the first (the intelligence test) 
but not the second. Psychology has a long and storied 
history (over one hundred years old) of measuring the 
intelligence trait. Although there has been psychological 
work on rational thinking, this research started much 
later and was not focused on individual differences. 

Many treatments of the intelligence concept could 
be characterized as permissive rather than grounded 
conceptualizations. Permissive theories include in their 
definitions of intelligence aspects of functioning that are 
captured by the vernacular term “intelligence” (adapta- 
tion to the environment, showing wisdom, creativity, 
etc.) whether or not these aspects are actually measured 
by existing tests of intelligence. Grounded theories, in 
contrast, confine the concept of intelligence to the set of 
mental abilities actually tested on IQ tests. Adopting per- 
missive definitions of the concept of intelligence serves 
to obscure what is absent from extant IQ tests. Instead, 
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in order to highlight the missing elements in IQ tests, we 
adopt a thoroughly grounded notion of the intelligence 
concept in this chapter—one that anchors the concept 
in what actual IQ tests measure. Likewise, we ground the 
concept of rationality in operationalizations from cur- 
rent cognitive science. 


2. Intelligence and Rationality in Cognitive Science 


The closest thing to a consensus, grounded theory of 
intelligence in psychology is the Cattell-Horn—Carroll 
(CHC) theory of intelligence (Carroll, 1993; Cattell, 
1963, 1998; Horn & Cattell, 1967). It yields a scientific 
concept of general intelligence, usually symbolized by g, 
and a small number of broad factors, of which two are 
dominant. Fluid intelligence reflects reasoning abilities 
operating across a variety of domains—including novel 
ones. It is measured by tests of abstract thinking such as 
figural analogies, Raven Matrices, and series completion. 
Crystallized intelligence reflects declarative knowledge 
acquired from acculturated learning experiences. It is 
measured by vocabulary tasks, verbal comprehension, 
and general knowledge measures. 

“Rationality” is a tortuous and tortured term in intel- 
lectual discourse (Stanovich, West, & Toplak, 2016). Many 
philosophical notions of rationality are crafted so as to 
equate all humans—thus, by fiat, defining away the very 
individual differences that a psychologist wishes to study. 
In contrast, rationality in the sense employed in cognitive 
science—and in this book—is a normative notion. Ratio- 
nality thus comes in degrees defined by the distance of 
the thought or behavior from the optimum defined by 
a normative model (Etzioni, 2014). Thus, when a cogni- 
tive scientist terms a behavior less than rational, he or 
she means that the behavior departs from the optimum 
prescribed by a particular normative model. 

We follow many cognitive science theorists in rec- 
ognizing two types of rationality, instrumental and 
epistemic (Manktelow, 2004; Over, 2004). The simplest 
definition of instrumental rationality is the following: 
behaving in the world so that you get exactly what you 
most want, given the resources (physical and mental) 
available to you. Epistemic rationality concerns how 
well beliefs map onto the actual structure of the world. 

More formally, economists and cognitive scientists 
define instrumental rationality as the maximization of 
expected utility. Expected utility is calculated by taking 
the utility of each outcome and multiplying it by the 
probability of that outcome occurring and then sum- 
ming those products over all of the possible outcomes. 
In practice, assessing rationality in this computational 
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manner can be difficult because eliciting personal prob- 
abilities can be tricky. Also, getting measurements of 
the utilities of various consequences can be experimen- 
tally difficult. Fortunately, there is another useful way 
to measure the rationality of decisions and deviations 
from rationality. It has been proven through several for- 
mal analyses that if people’s preferences follow certain 
consistent patterns (the so-called axioms of choice), 
then they are behaving as if they are maximizing util- 
ity (Dawes, 1998; Edwards, 1954; Jeffrey, 1983; Luce & 
Raiffa, 1957; Savage, 1954; von Neumann & Morgen- 
stern, 1944). These analyses have led to what has been 
termed the axiomatic approach to whether people are 
maximizing utility. It is what makes people’s degrees of 
rationality more easily measurable by the experimental 
methods of cognitive science. The deviation from the 
optimal choice pattern according to the axioms is an 
(inverse) measure of the degree of rationality. 

A substantial research literature—one comprising liter- 
ally hundreds of empirical studies conducted over several 
decades—has firmly established that people’s responses 
sometimes deviate from the performance considered notr- 
mative on many reasoning tasks. For example, people 
assess probabilities incorrectly, they test hypotheses inef- 
ficiently, they violate the axioms of utility theory, they do 
not properly calibrate degrees of belief, their choices are 
affected by irrelevant context, they ignore the alternative 
hypothesis when evaluating data, and they display numer- 
ous other information-processing biases (Baron, 2008, 
2014; Evans, 2014; Kahneman, 2011; Stanovich, 1999, 
2011; Stanovich et al., 2016). Much of the operational- 
ization of rational thinking in cognitive science comes 
from the heuristics and biases tradition, inaugurated by 
Kahneman and Tversky in the early 1970s (Kahneman & 
Tversky, 1972, 1973; Tversky & Kahneman, 1974). Thus, 
as measures of rationality, the tasks in the heuristics and 
biases literature, while tapping intelligence in part, actu- 
ally encompass more cognitive processes and knowledge 
than are assessed by IQ tests. In the next section, we will 
outline the functional cognitive theory that we will use 
to interpret the rational thinking tasks in this literature 
and show how they relate to intelligence. We will show 
that rationality is actually a more encompassing mental 
construct than is intelligence. 


3. Dual-Process Theory: The First Step toward 
a Model of Cognitive Architecture 


There is a wide variety of evidence that has converged 
on the conclusion that some type of dual-process model 
of the mind is needed in a diverse set of specialty areas 
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not limited to cognitive psychology, economics, social 
psychology, naturalistic philosophy, decision theory, 
and clinical psychology (Chein & Schneider, 2012; De 
Neys, 2018; Evans, 2008, 2010, 2014; Evans & Stanovich, 
2013; Sherman, Gawronski, & Trope, 2014; Stanovich, 
1999, 2004). Because there is now a plethora of dual- 
process theories (see Stanovich, 2011, 2012, for a list of 
the numerous versions of such theories), there is cur- 
rently much variation in the terms for the two processes. 
For the purposes of this chapter, we will most often 
adopt the Type 1/Type 2 terminology discussed by Evans 
and Stanovich (2013) and occasionally use the similar 
System 1/System 2 terminology of Stanovich (1999) 
and Kahneman (2011). The defining feature of Type 1 
processing is its autonomy—the execution of Type 1 
processes is mandatory when their triggering stimuli 
are encountered, and they are not dependent on input 
from high-level control systems. Autonomous processes 
have other correlated features—their execution tends to 
be rapid, they do not put a heavy load on central pro- 
cessing capacity, and they tend to be associative—but 
these other correlated features are not defining (Stano- 
vich & Toplak, 2012). 

In contrast with Type 1 processing, Type 2 processing 
is nonautonomous. It is relatively slow and computa- 
tionally expensive. Many Type 1 processes can operate 
at once in parallel, but Type 2 processing is largely serial. 
One of the most critical functions of Type 2 processing 
is to override Type 1 processing. This is because Type 
1 processing heuristics depend on benign environments 
providing obvious cues that elicit adaptive behaviors. 
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In hostile environments, reliance on heuristics can be 
costly (see Hilton, 2003; Over, 2000; Stanovich, 2004). 

Once detection of the conflict between the norma- 
tive response and the response triggered by System 1 
has taken place (De Neys & Pennycook, 2019; Stano- 
vich, 2018), Type 2 processing must display at least two 
related capabilities in order to override Type 1 pro- 
cessing. One is the capability of interrupting Type 1 
processing. The second is to enable processes of hypo- 
thetical reasoning and cognitive simulation that are a 
unique aspect of Type 2 processing (Evans, 2007, 2010; 
Evans & Stanovich, 2013). In order to reason hypotheti- 
cally, we must, however, have one critical cognitive 
capability—we must be able to prevent our representa- 
tions of the real world from becoming confused with 
representations of imaginary situations. The so-called 
cognitive decoupling operations (Stanovich, 2011; Sta- 
novich & Toplak, 2012) are the central feature of Type 2 
processing that make this possible, and they have impli- 
cations for how we conceptualize both intelligence and 
rationality, as we shall see. The important issue for our 
purposes is that decoupling secondary representations 
from the world and then maintaining the decoupling 
while simulation is carried out is a Type 2 processing 
operation. It is computationally taxing and greatly 
restricts the ability to conduct any other Type 2 opera- 
tion simultaneously. 

A preliminary dual-process model of mind, based 
on what we have outlined thus far, is presented in 
figure 15.2.1. The figure shows the Type 2 override func- 
tion we have been discussing, as well as the Type 2 process 
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Figure 15.2.1 


A preliminary dual-process model. 
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of simulation. Also rendered in the figure is an arrow 
indicating that Type 2 processes receive inputs from 
Type 1 computations. These so-called preattentive pro- 
cesses (Evans, 2008) establish the content of most Type 
2 processing. 


4. Differentiating Type 2 Processes: The Reflective 
and Algorithmic Minds 


In this section, we will explain why rational thinking 
stresses a level in the hierarchical control system of the 
brain that is only partly tapped by IQ tests. This is because 
the override mechanism depicted in figure 15.2.1 needs to 
be conceptualized in terms of two levels of processing— 
what are sometimes termed the algorithmic and reflec- 
tive levels of processing (Stanovich, 2011). 

Our attempt to differentiate the two levels of control 
involved in Type 2 processing is displayed in figure 15.2.2. 
The psychological literature provides much converging 
evidence and theory to support such a structure. First, 
psychometricians have long distinguished typical per- 
formance situations from optimal (sometimes termed 
“maximal”) performance situations (Ackerman & Kanfer, 
2004). Typical performance measures implicate, at least 
in part, the reflective mind—they assess goal prioritiza- 
tion and epistemic regulation. In contrast, optimal perfor- 
mance situations are those where the task interpretation 
is determined externally. The person performing the task 
is told the rules that maximize performance. Thus, opti- 
mal performance tasks assess questions of the efficiency 
of goal pursuit—they capture the processing efficiency of 
the algorithmic mind. All tests of intelligence or cognitive 
aptitude are optimal performance assessments, whereas 


Keith E. Stanovich, Maggie E. Toplak, and Richard F. West 


measures of critical or rational thinking are often assessed 
under typical performance conditions. 

The difference between the algorithmic mind and the 
reflective mind is captured in another well-established dis- 
tinction in the measurement of individual differences— 
the distinction between cognitive ability (intelligence) 
and thinking dispositions. The former are, as just men- 
tioned, measures of the efficiency of the algorithmic 
mind. The latter travel under a variety of names in psy- 
chology—thinking dispositions or cognitive styles being 
the two most popular. Examples of some thinking disposi- 
tions relevant to rationality that have been investigated 
by psychologists are actively open-minded thinking, need 
for cognition, consideration of future consequences, need 
for closure, and dogmatism (see Stanovich et al., 2016). 

In short, measures of individual differences in thinking 
dispositions are assessing variation in people’s goal man- 
agement, epistemic values, and epistemic self-regulation— 
differences in the operation of the reflective mind. People 
have indeed come up with definitions of intelligence that 
encompass the reflective level of processing, but never- 
theless, the actual measures of intelligence in use assess 
only algorithmic-level cognitive capacity. 

Figure 15.2.2 represents the classification of individ- 
ual differences in the tripartite view. The broken hori- 
zontal line represents the location of the key distinction 
in older, dual-process views. Figure 15.2.2 identifies 
variation in fluid intelligence with individual differ- 
ences in the efficiency of processing of the algorithmic 
mind. To a substantial extent, fluid intelligence mea- 
sures the ability to cognitively decouple—to suppress 
Type 1 activity and to enable hypothetical thinking. The 
raw ability to sustain such simulations while keeping the 
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relevant representations decoupled is one key aspect of 
the brain’s computational power that is being assessed 
by measures of fluid intelligence. This is becoming clear 
from converging work on executive function and work- 
ing memory, which both display correlations with fluid 
intelligence that are quite high (Duncan et al., 2008; 
Jastrzebski, Ciechanowska, & Chuderski, 2018; Stano- 
vich et al., 2016). This is because most measures of exec- 
utive function, such as working memory, are direct or 
indirect indicators of a person’s ability to sustain decou- 
pling operations. Thus, Type 2 processes are strongly 
associated with fluid intelligence. Finally, the reflective 
mind is identified with individual differences in think- 
ing dispositions related to beliefs and goals. 


5. Why Rationality Is a More Encompassing Construct 
than Intelligence 


Figure 15.2.2 highlights an important sense in which ratio- 
nality is a more encompassing construct than intelligence: 
as previously discussed, to be rational, a person must have 
well-calibrated beliefs and must act appropriately on those 
beliefs to achieve goals—both of these depend on the 
thinking dispositions of the reflective mind. The types 
of cognitive propensities that these thinking disposition 
measures reflect are the tendency to collect information 
before making up one’s mind, the tendency to seek vari- 
ous points of view before coming to a conclusion, the 
disposition to think extensively about a problem before 
responding, the tendency to calibrate the strength of one’s 
opinion to the degree of evidence available, the tendency 
to think about future consequences before taking action, 
the tendency to explicitly weigh pluses and minuses of 
situations before making a decision, and the tendency to 
seek nuance and avoid absolutism. 

In order to achieve both epistemic and instrumental 
rationality, an individual must also, of course, have the 
algorithmic-level machinery that enables him or her to 
carry out the actions and to process the environment in 
a way that enables the correct beliefs to be fixed and the 
correct actions to be taken. Thus, individual differences 
in rational thought and action can arise because of indi- 
vidual differences in fluid intelligence (the algorithmic 
mind) or because of individual differences in thinking 
dispositions (the reflective mind), or from a combina- 
tion of both. 

To put it simply, the concept of rationality encompasses 
thinking dispositions and algorithmic-level capacity, 
whereas the concept of intelligence (at least as it is com- 
monly operationalized) is largely confined to algorithmic- 
level capacity. Intelligence tests do not attempt to measure 
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aspects of epistemic or instrumental rationality, nor do 
they examine any thinking dispositions that relate to 
rationality. Thus, as long as variation in thinking dispo- 
sitions is not perfectly correlated with variation in fluid 
intelligence, there is the statistical possibility of rational- 
ity and intelligence explaining at least partially separable 
variance. 

In fact, substantial empirical evidence indicates that 
individual differences in thinking dispositions and intel- 
ligence are far from perfectly correlated. Studies (e.g., 
Ackerman & Heggestad, 1997; Cacioppo, Petty, Fein- 
stein, & Jarvis, 1996) have indicated that measures of 
intelligence display only moderate-to-weak correlations 
with some thinking dispositions (e.g., actively open- 
minded thinking, need for cognition) and near-zero cor- 
relations with others (e.g., conscientiousness, curiosity, 
diligence). Other important evidence supports the con- 
ceptual distinction made here between algorithmic cog- 
nitive capacity and thinking dispositions. For example, 
across a variety of tasks from the heuristics and biases 
literature, it has consistently been found that rational 
thinking dispositions will predict variance after the 
effects of general intelligence have been controlled (for 
a discussion and citations, see Stanovich et al., 2016). 


6. The Fleshed-Out Model 


The functions of the different levels of control are illus- 
trated more completely in figure 15.2.3. There, it is clear 
that the override capacity itself is a property of the algo- 
rithmic mind, and it is indicated by the arrow labeled A. 
However, previous dual-process theories have tended to 
ignore the higher-level cognitive operation that initiates 
the override function in the first place. This is a dispo- 
sitional property of the reflective mind that is related to 
rationality. In the model in figure 15.2.3, it corresponds 
to arrow B, which represents the instruction to the algo- 
rithmic mind to override the Type 1 response by taking 
it offline. This is a different mental function than the 
override function itself (arrow A), and the evidence cited 
above indicates that the two functions are indexed by 
different types of individual differences. 

The override function has loomed so large in dual- 
process theory that it has somewhat overshadowed 
the simulation process that computes the alternative 
response that makes the override worthwhile. Thus, 
figure 15.2.3 explicitly represents the simulation func- 
tion as well as the fact that the instruction to initiate 
simulation originates in the reflective mind. The decou- 
pling operation itself (indicated by arrow C) is carried 
out by the algorithmic mind. The instruction to initiate 
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A more complete model of the tripartite structure. 


simulation (indicated by arrow D) is carried out by the 
reflective mind. Again, two different types of individual 
differences are associated with the initiation call and the 
decoupling operator—specifically, thinking dispositions 
with the former and fluid intelligence with the latter. 
Also represented is the fact that the higher levels of con- 
trol receive inputs from the autonomous mind (arrow G) 
via so-called preattentive processes. 

The arrows labeled E and F reflect the decoupling and 
higher-level control of a kind of Type 2 processing (serial 
associative cognition) that does not involve fully explicit 
cognitive simulation, but we will not review its function 
here (see Stanovich, 2011). In figure 15.2.3, we can now 
identify a third function of the reflective mind: initiat- 
ing an interrupt of serial associative cognition (arrow F). 
This interrupt signal alters the next step in a serial asso- 
ciative sequence that would otherwise direct thought. 
This interrupt signal might stop serial associative cogni- 
tion altogether in order to initiate a comprehensive sim- 
ulation (arrow C) or start a new serial associative chain 
(arrow E) from a different starting point. 

Although taking the Type 1 response priming offline 
might itself be procedural, the process of synthesizing an 
alternative response often utilizes stored knowledge of 
various types. During the simulation process, declarative 


knowledge and strategic rules (linguistically coded strat- 
egies) are used to transform a decoupled representation. 
The knowledge, rules, procedures, and strategies that 
can be retrieved and used to transform decoupled repre- 
sentations have been referred to as “mindware,” a term 
coined by David Perkins in a 1995 book. The mindware 
available for use during cognitive simulation is in part 
the product of past learning experiences. 

Because the Cattell-Horn—Carroll (CHC) theory of 
intelligence is one of the most comprehensively vali- 
dated theories of intelligence available, it is important to 
see how two of its major components miss critical aspects 
of rational thought. Fluid intelligence will, of course, 
have some relation to rationality because it indexes the 
computational power of the algorithmic mind to sus- 
tain decoupling. Because override and simulation are 
important operations for rational thought, fluid intel- 
ligence will definitely facilitate rational action in some 
situations. Nevertheless, the tendency to initiate over- 
ride (arrow B in figure 15.2.3) and to initiate simulation 
activities (arrow D in figure 15.2.3) are both aspects of 
the reflective mind not assessed by intelligence tests, so 
the tests will miss these components of rationality. Such 
propensities are instead indexed by measures of typical 
performance (cognitive styles and thinking dispositions) 


Rationality and Intelligence 


as opposed to measures of maximal performance such as 
IQ tests. 

The situation with respect to crystallized intelligence 
is a little different. Rational thought depends critically on 
the acquisition of certain types of knowledge. That knowl- 
edge would, in the abstract, be classified as crystallized 
intelligence. But is it the kind of crystallized knowledge 
that is assessed on actual tests of intelligence? The answer 
is “no.” The knowledge structures that support rational 
thought are specialized. They cluster in the domains of 
probabilistic reasoning, causal reasoning, and scientific 
reasoning. In contrast, the crystallized knowledge assessed 
on IQ tests is deliberately designed to be nonspecialized. 
The designers of the tests, in order to make sure the sam- 
pling of vocabulary and knowledge is fair and unbiased, 
explicitly attempt to broadly sample vocabulary, verbal 
comprehension domains, and general knowledge. In 
short, crystallized intelligence, as traditionally measured, 
does not assess individual differences in rationality. 

Finally, there is one particular way the autonomous 
mind supports rationality that we would like to empha- 
size. It is that the autonomous mind contains rational 
rules and normative strategies that have been tightly 
compiled and are automatically activated due to over- 
learning and practice. This means that, for some people, 
in some instances, the normative response emanates 
directly from the autonomous mind rather than from 
the more costly Type 2 process of simulation. 


Mindware accessible 
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Figure 15.2.4 illustrates more clearly the point we 
wish to make here. This figure has been simplified by 
the removal of all the arrow labels and the removal of 
the boxes representing serial associative cognition, as 
well as the response boxes. In the upper right is repre- 
sented the accessing of mindware that is most discussed 
in the literature. In the case represented there, a nonnor- 
mative response from the autonomous mind has been 
interrupted, and the computationally taxing process of 
simulating an alternative response is under way. That 
simulation involves the computationally expensive pro- 
cess of accessing mindware for the simulation. 

In contrast to this type of normative mindware access, 
indicated in the lower left of the figure, is a qualitatively 
different way that mindware can determine the norma- 
tive response. The figure indicates the point we have 
stressed earlier: that within the autonomous mind can 
reside normative rules and rational strategies that have 
been practiced to automaticity and can automatically 
compete with (and often immediately defeat) any alter- 
native nonnormative response that is also stored in the 
autonomous mind (De Neys & Pennycook, 2019; Stano- 
vich, 2018). 

So it should be clear from figure 15.2.4 that it does not 
follow from the output of a normative response that Sys- 
tem 2 was necessarily the genesis of the rational respond- 
ing. According to the model just presented, rationality 
requires three different classes of mental characteristics: 
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first, algorithmic-level cognitive capacity (fluid intelli- 
gence) is needed for override and sustained simulation 
activities. Second, the reflective mind must be character- 
ized by the tendency to initiate the override of subopti- 
mal responses generated by the autonomous mind and 
to initiate simulation activities that will result in a bet- 
ter response. Finally, the mindware that allows the com- 
putation of rational responses needs to be available and 
accessible during simulation activities or be accessible 
from the autonomous mind because it has been highly 
practiced (see figure 15.2.4). 


7. Our Tripartite Cognitive Architecture Reconciles 
the Opposing Positions in the Great Rationality Debate 


Researchers working in the heuristics and biases tradi- 
tion tend to be so-called Meliorists (Stanovich, 1999, 
2004, 2010). They assume that human reasoning is 
not as good as it could be and that thinking could be 
improved. Thus, a Meliorist is one who feels that educa- 
tion and the provision of information could help make 
people more rational. This optimistic part of the Melior- 
ist message derives from the fact that Meliorists see a large 
gap between normative models of rational responding 
and descriptive models of what people actually do. Over 
the past several decades, an alternative interpretation of 
the findings from the heuristics and biases research pro- 
gram has been championed. Contributing to this alterna- 
tive interpretation have been philosophers, evolutionary 
psychologists, adaptationist modelers, and ecological the- 
orists (Cohen, 1981; Cosmides & Tooby, 1996; Gigerenzer, 
2007; Oaksford & Chater, 2007, 2012; Todd & Gigerenzer, 
2000). They have reinterpreted the modal response in 
most of the classic heuristics and biases experiments as 
indicating an optimal information-processing adaptation 
on the part of the subjects. This group of theorists—who 
argue that an assumption of maximal human rationality 
is the proper default position to take—have been termed 
the Panglossians. The Panglossian theorists often argue 
either that the normative model being applied is not the 
appropriate one because the subject’s interpretation of 
the task is different from what the researcher assumes it 
is or that the modal response in the task makes perfect 
sense from an evolutionary perspective. The contrasting 
positions of the Panglossians and Meliorists define the 
differing poles in what has been termed the Great Ratio- 
nality Debate in cognitive science—the debate about 
how much irrationality to attribute to human cognition 
(Gigerenzer, 1996; Kahneman & Tversky, 1996; Kelman, 
2011; Lee, 2006; Polonioli, 2015; Samuels & Stich, 2004; 
Stanovich, 1999, 2004; Stanovich & West, 2000; Stein, 
1996). 
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A reconciliation of the views of the Panglossians and 
Meliorists is possible, however, if we take two scientific 
steps. First, we must consider data patterns long ignored 
in the heuristics and biases literature: individual dif- 
ferences on rational thinking tasks. Second, we must 
understand the empirical patterns obtained through the 
lens of the modified and updated dual-process theory we 
outlined in this chapter. 

We have argued (Stanovich & West, 2000) that the sta- 
tistical distributions of the types of goals being pursued 
by Type 1 and Type 2 processing are different. Specifi- 
cally, there is a difference between the goals at the level 
of the gene and the goals at the level of the individual, 
and important consequences for the pursuit of rational- 
ity follow from this fact. The greater evolutionary age of 
some of the mechanisms underlying Type 1 processing 
accounts for why it more closely tracks ancient evolu- 
tionary goals (i.e., the genes’ goals), whereas Type 2 pro- 
cessing instantiates a more flexible goal hierarchy that 
is oriented toward maximizing overall goal satisfaction 
at the level of the whole organism. Type 2 processing 
(especially at the reflective level) is more attuned to the 
person’s needs as a coherent organism than is Type 1 
processing. As a result, in the minority of cases where 
the outputs of the two systems conflict, people will often 
be better off if they can accomplish a system override of 
the Type 1-triggered output (the full argument is con- 
tained in Stanovich, 2004). 

Instances when there is a conflict between the responses 
primed by Type 1 and Type 2 processing are thus inter- 
preted as reflecting conflicts between two different types 
of optimization: fitness maximization at the subpersonal 
genetic level and utility maximization at the personal 
level. A failure to differentiate these interests is at the heart 
of the disputes between researchers working in the heuris- 
tics and biases tradition and their critics in the evolution- 
ary psychology camp. First, it certainly must be said that 
the evolutionary psychologists are on to something with 
respect to the tasks they have analyzed, because in each 
case, the adaptive response is the modal response in the 
task—the one most subjects give. Nevertheless, this must 
be reconciled with a triangulating data pattern relevant to 
this discussion—an analysis of patterns of covariation and 
individual differences across these tasks. Specifically, we 
have found that cognitive ability often (but not always) 
dissociates from the response deemed adaptive from 
an evolutionary analysis (Stanovich & West, 1998, 1999, 
2000). 

The evolutionary psychologists are probably correct 
that most Type 1 processing is evolutionarily adaptive 
in the ancestral environment. Nevertheless, their evo- 
lutionary interpretations do not impeach the position 
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of the heuristics and biases researchers that the alterna- 
tive response given by the minority of subjects is ratio- 
nal at the level of the individual. Subjects of higher 
analytic intelligence are simply more prone to override 
Type 1 processing in order to produce responses that 
are epistemically and instrumentally rational. This rap- 
prochement between the two camps was introduced 
by Stanovich (1999), and subsequent research has only 
reinforced it (see Kahneman & Frederick, 2002; Kelman, 
2011; Samuels & Stich, 2004; Stanovich, 2004, 2011). 
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Summary 


Current perspectives on what constitutes “good thinking” 
suggest that it should include proper process (e.g., be inter- 
nally coherent/rational) and some pragmatic element of 
usefulness (e.g., it should produce actions and results that 
are accurate and/or beneficial to the thinker). Although 
people’s reasoning in economic and social domains has 
various shortcomings on both counts, it can be improved 
through training. Such training does not need to be inten- 
sive or time-consuming in order to produce measurable 
improvements in reasoning. However, it typically must go 
beyond simply telling people to be more rational, to “try 
harder,” or to avoid particular biases. The most effective 
training seems to be the type that uses diverse examples 
and offers practice and feedback opportunities. Although 
some research demonstrates correlational benefits of 
“good thinking” for various life outcomes, future work 
should examine whether training has causal effects on 
improving outcomes for people over time. 


1. What Is Good Thinking? 


Consider the following scenario: 


Imagine you spent $100 on a ticket for a ski trip to Michi- 
gan. Several weeks later you buy a $50 ticket for a ski trip to 
Wisconsin. You think you will enjoy the Wisconsin ski trip 
more than the Michigan ski trip. As you are storing your 
Wisconsin ski trip ticket in your wallet, you notice that both 
trips are for the same weekend. It’s too late to sell either 
ticket, and you cannot return either one. You must use one 
ticket and not the other. Which trip will you take? (adapted 
from Arkes & Blumer, 1985, p. 126). 


Which is the better decision? Why? 
Now, consider the following statements: 


If it’s raining, then the streets must be wet. 
It’s not raining. 
Therefore, the streets must not be wet. (Nisbett, 2015, p. 218) 


Is it better to agree or disagree with the conclusion? 
Why? 


Such exercises help us grapple with questions about 
what constitutes normative thinking. Are there bet- 
ter and worse ways of thinking? And if there are better 
ways of thinking, how can people improve? We begin 
this chapter with some definitions of good reasoning 
that have been popular across different cultures and dis- 
ciplines. We then summarize research on human judg- 
ment in the social sciences, focusing on diagnosis (“How 
good is human judgment?”) and potential solutions 
(“Can it be improved?”). 

Greek philosophers played a major role in defining 
“good thinking” in Western societies. Their rules of logic 
are used to this day. The second opening example of this 
chapter shows the commonly made “inverse error’”— 
reasoning that violates the rules of propositional logic. 
The first argument takes the form “If P then Q.” This 
means that if we observe P (rainy weather), we can con- 
clude Q (wet streets). The second statement says that 
we observe “not P.” Firm conclusions cannot in fact be 
drawn because P did not occur. 

Conclusions can be logically derived yet wrong, and 
they can be illogically derived yet right. The internal 
validity of an argument and its truth are two entirely 
separate things. Thus, modern psychologists’ defini- 
tions of good reasoning typically take both factors into 
account. One prominent perspective states that good 
reasoning should include “correspondence” and “coher- 
ence” (Hammond, 1996); its conclusions should gener- 
ally correspond with reality, and it should also use coherent 
processes to produce such conclusions. As another prom- 
inent psychologist put it, “Good judges should both ‘get 
it right’ and ‘think the right way’” (Tetlock, 2005, p. 7). 

Nevertheless, people in different societies have very 
different ideas about what constitutes proper, or nor- 
mative, reasoning. For example, dialectical systems of 
thought tend to emphasize the shifting nature of real- 
ity, acknowledging that reality is full of contradictions, 
changes, and coexisting yet opposing forces (DaMatta, 
1995; Nisbett, Peng, Choi, & Norenzayan, 2001; Peng & 
Nisbett, 1999). Correspondence and coherence may be 
hard or even impossible to achieve, since what is true at 
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one moment may be false a moment later. The dialec- 
tic system of thought is more heavily adopted in some 
East Asian and Latin American societies (de Oliveira & 
Nisbett, 2017a; Nisbett et al., 2001), and the difference 
may be partially traceable to differences in how philoso- 
phy developed. For example, in contrast to Greek phi- 
losophy, which was developed in the service of guiding 
debate, East Asian philosophy was developed to serve 
society. Thus, good thinking was not necessarily about 
being right in some logical abstract sense but about 
managing the vicissitudes of life in a pragmatically use- 
ful and socially harmonious way. 

Some Westerners have also argued that thinking 
should not be assessed merely by its adherence to abstract 
principles (Keys & Schwartz, 2007). Instead, one should 
consider how decision outcomes are experienced by the 
thinker and “how the decision-making process fits into 
the decision maker’s life as a whole” (p. 165). From an 
economic perspective, the correct answer to this chap- 
ter’s opening question about the ski tickets is to go on the 
trip that one would typically enjoy more since the cost 
of the $100 ticket cannot be recovered. Going on the less 
enjoyable, more expensive trip amounts to wasting time 
in addition to wasting money since one has thereby for- 
gone a more enjoyable way to spend one’s vacation. Yet 
most untrained people who answer that question choose 
the more expensive and less enjoyable trip to Michigan. 
Although that decision does not adhere to economic 
principles, choosing Wisconsin may make people feel 
like they are violating another cherished rule, “Waste not, 
want not” (Arkes & Ayton, 1999). The unpleasantness 
of violating that rule may carry over from the decision 
process into the decision experience, making Wisconsin 
the less appealing choice at the moment even if it is the 
more appealing choice when detached from the decision 
dilemma (Keys & Schwartz, 2007). 

To summarize, there are diverse perspectives on what 
constitutes good thinking across cultures and within 
societies. Most psychological research on judgment 
and decision making has studied Western thinking and 
evaluated it by Western standards of coherence and cor- 
respondence. Thus, our chapter adopts a similar Western 
focus, but we have reviewed important cultural variation 
in reasoning elsewhere (de Oliveira & Nisbett, 2017b; 
Yates & de Oliveira, 2016). 


2. How Good Is Human Judgment? 
Psychologists began to empirically study reasoning rela- 


tively recently. Initial studies from the 1960s through 
the 1980s suggested that although human judgment has 
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been adequate for survival, it is characterized by errors 
in logic and nonadherence to normative economic 
principles. In social judgment studies, people paid too 
much attention to the actor and not enough attention 
to the actor’s context (Ross, 1977). They based social 
judgments and frequency estimates not on base rates 
or valid predictors but on whether examples of similar 
cases easily came to mind (Tversky & Kahneman, 1973). 
In economic studies, people were loss-averse, leading 
them to make contradictory decisions when identical 
options were framed either as gains or as losses (Kahne- 
man & Tversky, 1982). People were also poor logicians. 
For example, they were terrible at testing conditional 
propositions. Wason’s card selection task tested how 
well people could falsify if P then Q propositions (Wason, 
1960). In the task, people received a rule (e.g., “If the 
front of the card has a vowel, then the back has an even 
number”). Participants would then examine four cards 
displaying letters and numbers (e.g., an A, a B,a 4, anda 
7). Their task was to turn over only as many cards as nec- 
essary to test whether the rule was violated. Few people 
would correctly turn over the A card and the 7 card. 

Compendiums of reasoning errors were published in 
books like Human Inference (Nisbett & Ross, 1980). Such 
works were not optimistic about human reasoning: “We 
insist that the errors demonstrated in the laboratory 
and chronicled in this book are the ingredients of indi- 
vidual and collective human tragedy” (p. 251). But after 
several years of experiments demonstrating reasoning 
errors, Nisbett and his colleagues failed to replicate one 
of Kahneman and Tversky’s findings. Specifically, for a 
problem in which people typically misapplied the law 
of large numbers, they found that a large share of their 
participants responded accurately (Nisbett, 1993, p. 4). 
Participants who had taken a statistics course did par- 
ticularly well, whereas those who had not taken statistics 
performed poorly, like Kahneman and Tversky’s original 
participants. 

Taking a different approach, another scholar found 
that although people were bad at the standard Wason 
card selection task, they were better at testing condi- 
tional propositions when the task mapped onto certain 
real-world scenarios. For example, when asked to turn 
over as many sales receipts as necessary to establish that, 
if the receipt is for more than $20, it has a signature on 
the back, people largely understood that they should 
check receipts with large amounts and unsigned backs 
(D’Andrade, 1982, as cited in Lehman, Lempert, & Nis- 
bett, 1988). 

These findings indicated that mental tools necessary 
to engage in sound reasoning were not altogether absent 
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from the human toolbox. Rather, they were neglected 
for certain problems, and absent concepts could per- 
haps be taught via formal training. Scholars hoping to 
improve cognition would need to be able to teach new 
strategies but also to find ways of encouraging the use 
of already existing strategies and rules across domains. 

In summary, there is ample research documenting 
reasoning “failures” in humans, but there are diverse 
views on just how much of a problem this is and on 
whether observations of such failures are partly due to 
how the questions are asked. Taking abstract logical 
tasks like Wason’s card selection task and couching them 
in more practical terms can improve people’s perfor- 
mance. Nevertheless, people—even experts—still make 
many errors in common everyday judgments. Judgment 
errors can materialize in social domains, in health care 
(e.g., Gigerenzer, Gaissmaier, Kurz-Milcke, Schwartz, & 
Woloshin, 2007), in economic domains (e.g., Arkes & 
Blumer, 1985), and in political domains (Tetlock, 2005). 
Although we are less gloomy about the state of reason- 
ing than psychologists were in the 1980s, there is sub- 
stantial room for improvement. 


3. Can Reasoning Be Improved? 


There have been many proposed approaches to improv- 
ing reasoning (for a general review, see Larrick, 2004). 
Approaches include changing the structure of the ques- 
tion or the task (e.g., Thaler, Sunstein, & Balz, 2012), 
using groups rather than individuals (Sunstein & Hastie, 
2015; Surowiecki, 2005), or even jettisoning the human 
altogether in favor of mathematical models (Dawes, 
1979). In this chapter, we focus on training. 

Consider the following problem (adapted from Gige- 
renzer et al., 2007): 


Assume you conduct breast cancer screening in a certain 
region. You know the following information about this 
region: (1) The probability that a woman has breast cancer 
is 1%; (2) if a woman has breast cancer, the probability that 
she tests positive is 90%; and (3) if a woman does not have 
breast cancer, the probability that she nevertheless tests pos- 
itive is 9%. Now imagine that a woman tests positive. What 
are the chances that she has breast cancer? 


Laypeople and professionals struggle with this type of 
problem. In a survey of 160 gynecologists, only 21% 
responded correctly when given four answer choices 
(Gigerenzer et al., 2007). (She has a 10% chance; out of 
10 women with a positive mammogram, about 1 has can- 
cer.) Those doctors received a training session in which 
they were taught to convert percentages into frequencies 
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(e.g., think of 10% as 1 in 10, or 10 out of 100). After- 
ward, 87% were able to give the correct answer. 

Research on training has generally found that rea- 
soning can be improved through training and that 
this training does not need to be terribly extensive. 
We review some representative studies below that also 
address additional questions: Are some types of training 
more effective than others? How long do effects last? 
Can training on one rule in one domain transfer such 
that those newly learned skills are more broadly applied 
to other problems? We begin with some findings from 
statistical reasoning and then report some results on 
how the environment and feedback can improve ratio- 
nal reasoning. 


3.1 Learning Statistics 

Although early psychologists were skeptical about the 
effectiveness of training, research on training has pro- 
duced encouraging results. For example, one study 
assessed people’s reasoning by asking them to explain 
why a traveling saleswoman’s second visit to a restaurant 
typically leads to disappointment when her first meal 
had been outstanding (Fong, Krantz, & Nisbett, 1986). 
On such questions, respondents with no statistical back- 
ground gave exclusively nonstatistical answers (e.g., 
suggesting that the chefs change a lot or that her expec- 
tations were too high). Respondents who had taken one 
statistics course gave statistical answers about 20% of the 
time (e.g., suggesting she was lucky the first time, noting 
that restaurant meal quality varies). New graduate stu- 
dents in psychology gave statistical answers about 40% 
of the time, and doctoral-level scientists at a research 
institution gave such answers about 80% of the time. 
Giving a statistical answer to questions such as these 
reflects better thinking because single extreme observa- 
tions are typically followed by less extreme observations; 
extreme events “regress” toward the mean value more 
often than not. 

Further research suggested that the effects of for- 
mal training on reasoning depended on how much the 
program linked abstract principles, including statisti- 
cal ones, to everyday life problems. Skills taught in a 
domain-restricted manner may show less generalizabil- 
ity than skills taught with examples of diverse applica- 
tions. Lehman and colleagues (1988) found that getting 
a graduate degree in medicine or psychology was associ- 
ated with substantial increases in conditional reasoning 
and statistical/methodological reasoning, but getting a 
graduate chemistry degree was not. Chemistry students 
typically study very little statistics or scientific method- 
ology with direct relevance to everyday life. Psychology 
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and medical students study a great deal of statistics and 
scientific methodology having relevance to everyday- 
life domains. 

This idea—that people can generalize learned reason- 
ing rules across domains if they are shown how and get 
practice examples with it—was tested in a more formal 
experiment (Fong et al., 1986). In their first study, the 
researchers tested people’s reasoning through problems 
that varied from seeming relatively objective (e.g., judg- 
ing the characteristics of a lottery) to relatively subjec- 
tive (e.g., choosing which college to attend). Participants 
were randomly assigned to one of five conditions. In 
one condition, people read a packet explaining statisti- 
cal principles such as the law of large numbers. (The law 
of large numbers in statistics states that smaller samples 
are more likely than larger samples to vary from the 
population distribution.) After reading the packet, they 
watched the researcher demonstrating the principle by 
drawing balls from an urn. In another condition, partici- 
pants worked through example problems in which those 
abstract principles were applied to everyday problems. 
For example, they read about hypothetical ballet audi- 
tions and learned to think of the observed dance moves 
as a sample of a larger “population” of dance moves. 
They were reminded that smaller samples—like dance 
moves observed at an audition—are unreliable estimates 
of population distributions—such as all of a dancer’s 
moves. They then considered how, based on this obser- 
vation, it would make sense that some of the best dancers 
at an audition would turn out to be average performers 
after all; their good performance at the audition was a 
statistical fluke due to small sample size. In a third con- 
dition, participants read the informational packet and 
worked through the example problems. There were two 
control conditions; in one, participants were given no 
training. In another, they were simply given the defini- 
tions of each of the statistical principles. 

Training influenced the frequency and quality of 
statistical responses. In both control conditions, fewer 
than half of the responses invoked statistical princi- 
ples, and the quality of the statistical answers they gave 
was low. In the full training condition, about 64% of 
responses invoked statistical principles, and the quality 
of the answers was relatively high. The examples-only 
training and the packet-only training conditions fell in 
between. 

A similar study tested whether training in one domain 
generalized to other domains and whether effects per- 
sisted over time (Fong & Nisbett, 1991). Participants 
were trained on the law of large numbers through exam- 
ple problems. They were tested immediately or after a 
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two-week delay, either in the same domain as the exam- 
ples or in a different domain. The training was equally 
effective immediately whether participants were tested 
in the same or a different domain. Participants tested 
later in the same domain showed no decline in their per- 
formance. Participants tested later in a different domain 
showed some decline, but they still performed better 
than participants who had received no training. Simi- 
lar work examining financial training found domain- 
independent gains as well: people trained in normative 
economic reasoning with financial (vs. nonfinancial) 
examples did better on both financial and nonfinan- 
cial measures of reasoning (Larrick, Morgan, & Nisbett, 
1990), and training gains were still observed four to six 
weeks post-training. It’s notable that both statistical 
training and cost-benefit analysis training improved 
answers to questions in non-laboratory, nonacademic 
settings such as alleged telephone surveys about politics 
or consumer preferences. 


3.2 The Learning Environment and Feedback 
Competence is best developed in contexts where the 
learners can get relatively fast, frequent, and complete 
feedback on their performance—this allows people to 
know whether there is a problem with their reasoning in 
the first place (Hogarth, 2001; Larrick, 2004). For exam- 
ple, if one wants to know how good one is at spotting 
academic talent in the graduate admissions process, one 
would need to track outcomes of one’s decisions—both 
the accepted and rejected students—over many admis- 
sions cycles. Some real-world tasks provide people with 
feedback-heavy learning environments (e.g., weather 
forecasting, sports training), but many tasks provide 
slow and incomplete feedback (e.g., admissions and hir- 
ing decisions). 

Training programs can artificially structure the learn- 
ing environment so that it provides fast and complete 
feedback on tasks that otherwise are learned in feedback- 
poor real-world contexts. Morewedge and colleagues 
(2015) compared the effectiveness of video training and 
video game training for improving reasoning. The video 
game allowed participants to not only learn about the 
biases but also engage in problem-solving tasks that 
elicited those biases. After responding to the task, they 
received feedback on their performance as well as sugges- 
tions for how to improve. Participants trained with vid- 
eos merely received information about those biases and 
did not receive performance feedback. The interventions 
targeted bias blind spot (i.e., thinking you are less biased 
than others), confirmation bias (i.e., overweighting con- 
firming vs. disconfirming evidence), the fundamental 
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attribution error (i.e., neglecting situational influences 
when attributing events to a person), anchoring (i.e., 
overweighting initial information when making a sub- 
sequent judgment), the representativeness heuristic (i.e., 
relying on judgments of similarity to make inferences 
about probability), and social projection (i.e., assuming 
others’ emotions and values are like one’s own). 

In both studies, comparisons of pre- and post-test 
reasoning revealed that both the video and the video 
game reduced bias. Follow-up assessments two to three 
months later showed that the improvements were sus- 
tained for both methods. However, the video game debi- 
asing effects were much larger than the video effects at 
both post-intervention assessment times. These results 
suggest that personal feedback and practice greatly 
enhance the effects of training. 

One research team recently demonstrated increases 
in geopolitical forecasting competence both by training 
people with instructional modules and also by placing 
workers in more interactive, feedback-rich team environ- 
ments (Mellers et al., 2014). The training participants 
received either 45-minute “scenario training” or “prob- 
ability training” of similar length. The “scenario training” 
module taught forecasters to generate new futures, actively 
entertain more possibilities, use decision trees, and avoid 
certain biases. The “probability training” module taught 
forecasters to average multiple estimates from models, 
polls, and expert panels; extrapolate over time when 
variables were continuous; and avoid errors like base-rate 
neglect. Forecasters working in teams (vs. individually) 
had ample experiential learning opportunities; they could 
make forecasts as often as they wished, interact with other 
forecasters, share information via a web-based platform, 
get feedback on their thoughts and on their forecasts, and 
learn from that feedback. Training improved performance 
in both the first year and the second year of the tourna- 
ment. Teaming also helped: forecasters working in teams 
did better than those working individually. Remarkably, 
although the authors did not offer formal statistical analy- 
ses on the matter, the benefit of teaming seemed to be 
larger than that of training. 


4. Conclusion 


Training can improve reasoning, at least by Western def- 
initions, and the type of training matters. Training with 
examples and practice helps people generalize specific 
skills to a variety of domains, and feedback-rich envi- 
ronments likely boost training effects and help maintain 
gains over time. Programs do not need to be intensive 
or long in order to produce lasting effects on reasoning, 
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but they do need to go beyond merely telling people 
about common judgment errors or instructing them in 
abstract principles. 

There is good reason to believe training effects would 
carry over to everyday life problems that do not closely 
resemble problems used for training. Research has linked 
higher normative reasoning ability to lower risky behav- 
ior and juvenile delinquency (Parker, Bruine de Bruin, 
Fischhoff, & Weller, 2018) as well as higher salaries for 
a sample of professors and higher GPA for a sample of 
students (Larrick, Nisbett, & Morgan, 1993). 
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belief base, 11, 32, 328 
belief bias, 5-6, 8, 36, 88, 92, 175, 178, 772 (see also Bias) 
belief function, 20, 311-312, 341 
belief set, 11, 32, 338 
change, 340-342 (see also Belief revision) 
change, iterated, 331-332, 340 
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common belief in rationality, 545, 767, 772 
conditional, 339 
contraction, 329, 330-331, 340, 342 
correct, 548, 580, 583 
degree of, 6, 11-12, 19, 239-240, 278, 279, 282, 337, 397, 
582 
dispositional, 40, 338, 439, 444-446 
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higher-order, 38, 544 
joint, 28-29 
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non-Bayesian representation, 496, 500-501 
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prior belief and knowledge, 23, 37, 347, 349-350, 440, 443, 
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and probability, 12 
rational, 495, 608 (see also Decision, rational) 
representation, 328 
representation, non-Bayesian, 496, 500-501 
revision (see Belief revision) 
strong belief in rationality, 549 
and truth, 16-17 
update (see Belief revision) 
Belief revision, 40, 45, 46, 327-335, 340-342, 347, 350-356, 
400-401, 450, 549 
conservative, 331 
dynamic logic of, 189 
external, 331 
in human reasoning, 347, 350-352, 353, 354, 355 
iterated, 331-332, 340 
moderate, 331 
multiple, 332 
non-prioritized, 332 
radical, 331 
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Best-response dynamics, 562, 647 
Bias, 160-162, 436-437, 518-521, 633, 640, 641, 751, 752, 
758, 759, 762 
belief bias in human reasoning, 5-6, 8, 36, 88, 92, 175, 178, 
772 
blind spot, 804 
confirmation, 29, 638, 639, 771, 772, 804 
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sampling, 490, 491 
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Biological naturalism, 9 
Boethius’ Theses, 280 
Bounded rationality, 6, 15, 24, 34, 89, 92, 164-165, 322, 490, 
505-513, 518, 520, 521, 523, 573, 569, 572, 574, 625, 728, 
768-769 
Brain, 35, 117-125, 736, 739, 740. See also Cortex; Lobe 
damage, 36 
imaging, 35, 36, 37, 49, 739, 740 
lesion, 36 
regions associated with decision making, 36 
regions associated with reasoning, 36, 739, 740 
volume and cognitive abilities, 35 
Bridge assumption, 612 
Bridge principle, 206 
Brier score, 242 
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Burden of proof, 675 
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Capacity (in the mathematical sense), 306, 500 
Case-based reasoning, 672, 673 
Categorical imperative, 9, 27, 45-46, 48, 79, 80. See also 
Imperative, hypothetical 
Causal 
attribution, 443, 452 
inference, 279, 411, 413, 439-446, 451 
model, 371, 374, 413, 439-444, 451, 702 
network, 429-435, 439-444, 451 
power, 441, 442, 444, 451 
reasoning, 104-105, 371, 374, 411-413, 439-446, 451, 786 
responsibility, 452 
structure, 431, 441, 443, 451 
Causation, causality, 105, 427-437, 439-446, 449, 537 
direct and indirect, 433 
Caution, 498 
Centering inference, 399-400 
Chance (objective), 240. See also Probability, physical 
Change, minimal, 153, 330, 332, 350 
Cheater detection, 42, 103, 106, 108, 639, 640, 783 
Children, 104-105, 107, 780-787 
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Choice, 138-139, 485 
complex risky choice task, 489 
rational, 6, 28, 77, 473, 505, 625, 647 (see also Decision theory) 
resolute, 462, 479, 690 
risky, 474-482, 485, 486, 487, 488, 510, 636, 637 
sequential, 478 
social, 29, 332, 580, 583, 595-597 
sophisticated, 479 
theory choice, 748, 749-750, 760, 761 
Church-Turing thesis, 31 
Civil law, 661-662. See also Criminal law 
Closed world assumption, 8 
Coevolution, 101 
gene-culture, 103 
socio-cognitive, 104 
socio-linguistic, 103 
socio-technical, 104 
Cognitive 
illusion, 162, 166 
labor, 101, 103, 108, 767-768 
limitation, 162, 489, 505-506, 509, 636 
miserliness, 517, 520, 521, 522, 523 
modelling, 491, 729-730, 740-741 
success, 24-25, 101, 511 
variability, 651, 653-654 
Cognitive science, 31, 33, 50, 51 
Coherence, 130-131, 165, 278-281, 328-331, 399, 519, 520, 
523, 630, 801 
dispositional, 330, 331 
dynamic, 330, 333 
interval, 278, 281, 399 
probabilistic, 278-281, 286-287 
static, 278, 329 
Coleman’s diagram, 611-612, 613, 614, 616, 618, 619 
Collaboration, 27, 28, 42-43, 101, 103-104, 615, 627, 638-641 
Collective 
good, 612, 613-615 
intentionality, 597-598, 609 
knowledge, 580 
rationality, 28-31, 580, 585, 591-598, 627 
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Combinative preference, 461 
Commitment, 328, 539-540, 598 
Common difference effect, 571, 574 
Common knowledge, 321, 540, 603 
Communication, 290-293 
Gricean account of, 29, 45, 601-603 
normality conditions of, 603 
Communicative action, 601, 603, 606, 608 
Communicative situation, perfect, 604 
Comparative theorizing, 11, 12, 15 
Compensatory strategy, 164 
Competence, 21, 25, 328 
vs. performance, 1, 23, 25 
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Completeness, 204, 331, 429, 431, 462 
Complex cognition, 31, 35, 38 
Complexity, 508, 729, 768 
complexity theory, 34 
Comprehensive rationality, 627 
Computation, 185, 186, 187, 443 
Computational level of explanation, 9, 13, 15, 31. See also 
Rationality, output-oriented 
Computational models of syllogistic reasoning, 
229-233 
Computer simulation, 684-685, 689 
Computing, distributed, 186, 190 
Concepts, 41, 51 
Conceptual-role semantics, 216, 223 
Conditional 
independence, 254, 267, 340, 430-431 
preference, 470 
probability, 108, 239, 265-274, 278-279, 296, 308, 353, 354, 
389, 407-409, 441, 442, 582, 636, 640-641 
probability hypothesis, 95, 286, 296-300, 302, 353, 395-397, 
399-400 
proposition, 802 
reasoning in psychology, 38, 43, 108, 177, 348, 349-350, 
405-410, 637-640, 784-786 
Conditionals, 20, 38, 95, 266, 273, 277-282, 295, 342, 343, 
348, 349-355, 395, 405-415, 419-422, 427, 802 
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indicative, 266, 273, 279, 381, 405-410 
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material, 52, 216, 216, 218, 222 
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probabilistic analyses of, 389, 398 
restrictor analysis of, 387 
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subjunctive, 391, 536, 539, 410-415 
transitivity of, 281, 384 
truth-functional analysis of, 277, 383 
utility conditionals, 11, 419-422, 473-477 
variably strict, 386 
Conditioning/Conditionalization, 13, 244, 340, 
427-428 
Bayesian, 287, 400, 427-428, 633, 636, 641 
Jeffrey, 245, 287, 344, 401 
Condorcet Paradox, 463 
Confidence, 501-502, 517-523. See also Overconfidence 
confidence judgment, 38, 306-307 
measure, 306-307 
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Confirmation, 148-149, 266-273 
confirmation bias, 29, 638, 639, 771, 772, 804 
hypothetico-deductive model of, 747-748 
incremental, 266-273, 339 
Conflict 
normative, 648-649 
resolution, 36 
Conformity, 30, 586 
Conjunction fallacy, 10, 91, 154, 273, 281, 399 
Connective, logical, 202, 277 
Connectivity of neurons, 35 
Connexive logic, 278-280 
Consciousness, 23, 32-33, 40, 42 
Consensus, 584, 772 
Consequence. See Inference 
logical, 203, 205, 367, 381, 715, 717 
unintended, 611, 618, 619 
Consequent, 295, 381. See also Affirming the consequent; 
Antecedent 
Consequentialism, 48, 536, 537 
Conservativity (norm), 256, 330, 332 
Consistency, 33, 39, 40, 116-117, 154-156, 203, 206. See also 
Inconsistency 
Constitutivism (concerning reasons for action), 699-700 
Constrained maximization, 593, 685, 690 
Constructivism, 17, 580 
Content, 11, 14, 51, 349, 351, 731 
content effects in human reasoning, 5, 36-37, 41, 154, 638, 
639 
content-specific rule, 638, 639, 652 
variation, 783, 785-786 
Context, 350 
of discovery, 82, 747, 758, 759 
of justification, 82, 747, 758, 759, 761 
sociopolitical, 759, 760 
Continental philosophy, 49-50 
Continuity effect, 728 
Contractarianism, 48, 687, 689 
Contraposition, 281, 384 
Contrary-to-duty obligation, 648 
Convention, 46, 593, 604 
Conversation, 286 
Conversational implicature, 45 
Conversational pragmatics, 653 
Cooperation, 27, 28, 42-43, 101, 103-104, 591-594, 604, 615, 
627, 638-641 
Cooperative game theory, 27, 45, 529, 540, 685-686, 690 
(see also Game theory, non-cooperative) 
Cooperative principle, 45 
Coordination, 593 
Correctness, 129, 143-144, 319, 580, 715, 720 
Correspondence, 165, 801 
Cortex, 35-37. See also Brain; Lobe 
frontal, 35, 37 
occipital, 36 
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parietal, 36, 739, 740 
prefrontal, 35, 36, 37 
temporal, 36 
visual, 36, 739, 740 
Costs and benefits, 636-641, 653 
Counterfactual(s), 279, 327, 381, 405, 430-436, 538-539 
computation of, 429 
counterfactual inference effect, 411 
in human reasoning, 33, 36, 42, 105, 154, 161, 219, 279, 
398, 405, 410-415 
semantics of, 429 
structural, 430 
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paradigm 
Cox axioms, 310 
Credence, 239-248, 265-267 
Credibility, 421, 539-540, 637-638, 771 
Criminal law, 662, 664. See also Civil law 
Critical evaluation, 627 
Cue utilization, 23, 163 
Cultural relativity, 41 
Cumulation principle, 488 
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Decision, 659-666 
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costs, 34 
from experience, 489, 490, 491 
non-Bayesian decision model, 496-497, 500-502 
policy decision, 757, 761 
rational, 495, 497-499 (see also Belief, rational) 
under uncertainty, 19, 495-503, 506, 513, 636-637, 640-641 
Decision making, 93-94, 801-802, 804 
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cognitive theories of, 6, 10, 23-24, 162-163 (see also 
Bounded rationality; Prospect theory) 
emotion-based, 702, 704 
errors in, 23, 24 
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judgment and, 10, 23, 30 (see also Decision theory) 
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Decision theory, 6, 15, 19, 140-141, 333, 368, 427, 473-484, 
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sion making; Rational choice 
causal, 427, 479-480 
descriptive, 485, 487, 491, 502 
dynamic, 16, 477, 498 
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evidential, 479-480 
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Declarative memory, 215-216 
Decreasing abstraction, 614 
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Deduction, 120-123, 148-150, 381, 669-670, 735, 736, 
638-741 
natural, 203, 206 
Deductive reasoning, 4, 13, 19, 33, 72, 87-89, 91-92, 106, 
148-150, 405-410, 715-722. See also Dual-process model/ 
theory; Mental, logic; Mental models (theory) 
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Defeasible reasoning, 5, 8, 13, 333, 347, 348-350, 352-356, 
361, 649, 669-672 
Deference, 769-771 
de Finetti truth table, 398 
Degrees of belief, 6, 11-12, 19, 239-240, 278, 279, 282, 337, 
397, 582 
Degrees of rationality, 140, 143 
Deliberation, 228-234, 489, 490, 491, 521, 522, 584 
practical, 693-700 
social context of, 699 
Demarcation, scientific, 767-768 
Demonstrability (of a shared belief), 29 
Demonstratio (apodeixis), 73, 79. See also Intuitio (notis) 
Dempster-Shafer belief functions, 20, 311-312, 341 
Denying the antecedent (DA), 296, 343, 348, 349, 406 
Deontic 
logic, 639-640, 645, 646, 647, 648, 649, 651 
reasoning, 638-640, 649, 651-655 
selection task, 640, 651-653 
speech act, 653, 654 
Deontologism, 48 
Description—experience gap, 489, 490 
Descriptive decision theory, 485, 487, 491, 502 
Descriptive-normative divide, 71, 73, 77, 78, 79, 80, 81, 82. 
See also Normative theory/theorizing 
Desire, 8-9, 14-16, 142, 681-689, 695, 699 
extrinsic, 9, 14, 15 
fulfillment, 682-689, 689 
implicit, 682 
instrumental, 694, 695 
intrinsic, 9, 682, 694-700 
moral, 682-684 
morally causal, 683-685 
past, present, and future desires, 685 
rationality of, 8-9 
second-order, 38 
Development 
of argumentation, 780 
metalogical, 779, 780 
of rationality, 779, 780-787 
of reasoning, 781-787 
Diachronic theorizing. See Dynamics 
Diagnostic inference, 443, 450 
Diagnostic reasoning, 443, 449 
Diagram, 429-435, 735, 736 
causal, 431-436 
selection diagram, 434 
Diagrammatic reasoning, 715-720 
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Dialectic thinking, 50, 801 
Dialogue logic, 370 
Dictatorship (social choice theory), 30 
Difference measure, 270-273. See also Relevance, measure 
Difficulty in reasoning, 226, 228-229, 234 
Dimension (in legal reasoning), 672 
Diminishing 
criterion model, 521, 523 
marginal utility, 465, 486, 487 
sensitivity, 485, 486, 491 
Directed acyclic graph (DAG), 253, 429-435, 441, 451 
Direction of fit, 10 
Directive, 208 
Disabler/Disabling conditions, 348, 349-351, 354. See also 
Alternative condition 
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Discounting 
exponential, 469 
hyperbolic, 469 
Discursive dilemma, 581 
Disjunction, 148-150, 271, 273, 274, 277, 408-410 
Disposition(s), 32-33, 52, 439, 444-446 
preferences as behavioral, 536 
Dissociation, double, 36, 122 
Distributed computing, 186, 190 
Distributed knowledge, 322 
Divergent thinking, 786, 787 
Diversity, 763 
Division of cognitive labor, 101, 103, 108, 767-768 
do-calculus, 430-431 
Domain-specific vs. domain-general, 439, 440, 443, 
638-640 
Dominance, 210. See also Strategy, strictly dominated; Strategy, 
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do-operator, 413, 428, 442 
Double dissociation, 36, 122 
Doxastic 
logic, 319, 337 
preference, 329-330, 331 
voluntarism, 11, 333 
d-separation, 430 
Dual-process model/theory, 22-23, 49, 93-95, 149-150, 167, 
173, 174, 175, 176-181, 229, 231, 233, 301, 354, 355, 616, 
702, 792 
Dual-source model, 177, 301 
Dual-strategy model, 108, 177 
Dunning-Kruger effect, 518, 519 
Dutch book argument, 151, 240-241, 278, 399 
Dynamical system, 191, 192 
Dynamic-epistemic logic, 189 
Dynamic game, 549 
Dynamics, 12-14, 15 
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externally driven, 13, 15, 32 
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Easiness effect, 771 
Ecological rationality, 7, 93, 165, 510-512, 636 
Ecological validity, 24, 101, 163, 636-637 
Economic rationality, 330-331, 625, 630, 639-641 
Economics, 3, 6, 28, 331 
behavioral, 22, 37, 618, 640-641 
mindless, 467 
Egoism, 24, 28, 43, 683-684, 689, 694. See also Altruism 
Elimination, iterated (of dominated strategies), 
546 
Elisberg paradox, 19-20, 26, 311, 489, 495-496 
Embeddedness, 617 
Embodiment, 413-415 
Emotion-based decision making, 702, 704 
Emotions and rationality, 7, 47, 568, 572 
Empiricism, logical, 20, 52 
Endoxa, 73 
Enkrasia, 129, 134 
Enlightenment, 16, 44, 71, 74, 772 
Entailment, 266-267, 272, 367 
Entrenchment, epistemic, 13, 330, 331, 338, 350 
Entropy, 244 
relative, 245 
Environment, 506, 508, 510-511 
virtual, 737 
Epistemic 
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authority, 769-771 
democracy, 580-581 
entrenchment, 13, 330, 331, 338, 350 
epistemic conditions for Nash equilibrium, 567 
game theory, 27, 529, 540, 543 
injustice, 586 
logic, 305, 308, 319 
model, 544 
rationality, 17, 29, 44-45, 46, 176, 518, 519, 523, 694-695, 
735, 792 (see also Rationality, theoretical) 
reasoning, 29, 769-772 
skepticism and nihilism, 586 
status, 412 
trust/trustworthiness, 351, 769, 770, 771 
uncertainty, 768-769 
utility, 209 
value, 748, 749-750, 760, 761 
Epistemology, 24-25, 328 
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moral, 705 computational level of, 9, 13, 15, 31 (see also Rationality, 
social, 30, 579-580, 750 output-oriented) 
of visualization, 736-741 implementational level of, 9 
Equation, the. See Conditional, probability hypothesis inference to the best, 45, 291 
Equilibrium. See Reflective equilibrium Explanatory depth, illusion of, 518, 770 
beliefs about other players and equilibrium, 537-539, Exponential discounting, 469 
540-541, 544, 547, 567 Extensive game, 189, 190, 529-531, 553 
epistemic conditions for Nash equilibrium, 567 Externalism, mental, 132 
equilibrium selection, 553 External validity, 91-92, 434 
Nash, 27-28, 532-533, 547, 554, 613-615, 647, Eye-tracking, 491 
690 
strict Nash, 532, 533, 539, 554 F 
subgame-perfect, 533-535 Factivity, 321 
Error(s) Factor (in legal reasoning), 672, 673-674 
in cognitive theories, 5, 8, 13, 18, 22-23 Faculty of reason, 72, 74, 75-80, 130 
in decision making, 23, 24 Fallacy, 154, 370-371, 372, 399 
fundamental attribution error, 804-805 conjunction fallacy, 10, 91, 154, 273, 281, 399 
in human reasoning, 8, 13, 22-23, 738-739, 802 False-belief task, 39, 43, 104 
inverse, 801 Falsificationism, 5, 90, 748 
error probability, 761-762 Fast and frugal heuristics, 15, 93, 163-164, 488, 489, 491, 
in syllogistic reasoning, 226, 228, 232, 234 510-512, 572 
Ethics. See Philosophy, moral Feedback, 804-805 
Evaluation Feeling of rightness (FOR), 38 
vs. appraisal, 210 Feminism, 758, 760 
critical, 627 Feminist objectivity, 751 
phase, 488 Feminist philosophy of science, 751 
Evidence, 131, 132, 265-274 First-order logic, 204, 633, 637-640 
evidence accumulation models, 491 Fluid intelligence, 792 
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problem of old, 268 Folk psychology, 741 
for rationality, 570-574 Folk theorem, 684-685, 689 
weight of, 268-273, 495, 500-501, 804 FOR. See Feeling of rightness 
Evolution, 101 Force, illocutionary, 605 
biological, 102, 633-635 Forecasting, 805 
cultural, 102 Forgetting, 246 
individual, 102 Formalization, 277, 614, 717, 718 
Evolutionary game (theory), 27, 43, 103, 191, 529, 541, 555, Formal operation phase, 90, 782 
638, 639-641 Forward induction, 539, 549 
dynamics of, 556 Fractionated system, 117 
Evolutionary rationality, 42, 92-93, 101, 633-641 Framing effect, 636-637 
Exception, 350, 421 Free choice permission, 648 
Exclusionary preference, 461 Freedom of the will, 38 
Executive process, 36 Free rider problem, 612 
Expectation, 143, 333 Frontal lobe, 117-124 
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extensive, 189, 190, 529-531, 553 
logic of, 189 
game tree, 529-531 
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Game theory, 27, 28, 43, 148, 647, 648, 685-686, 690 
applied, 535-541, 638, 639-641, 675-676 
Bayesian, 27, 532 
behavioral, 529, 541, 568, 620 
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evolutionary, 27, 43, 103, 529, 541, 638, 639-641 
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Hypothetico-deductive method, 5 
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Ignorance, 319 
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Imaging (vs. conditionalization), 428 
Impartial pairwise comparison dynamics, 559, 562 
Imperative 
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Inducement and advice conditional, 653 
Induction, 13, 102, 106, 117-118, 152-153, 720 
backward, 479, 534-535, 538, 549, 639 
forward, 539, 550 
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Lewis-style conditional, 386, 397, 427-428 validity, 147, 149, 203, 205, 225, 278, 279-281, 362, 381, 
Lexicographic strategy, 165 406-407, 720 


Liability, strict, 134 Logicism, 4, 81 


Subject Index 


Log-likelihood ratio, 269-274 

Logos, 3, 71, 72, 74, 82. See also Noiis 
Long-term memory, 21, 33, 39-40, 354 
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relevance measure, 268-274 (see also Likelihood, ratio; 

Relevance, quotient) 

Z measure of relevance, 270-271, 274 
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in temporal reasoning, 726, 728 
Mental state 

conscious, 23, 32-33, 40, 42 

occurrent vs. dispositional, 32-33, 40 
Meta- 

analysis of reasoning, 225-227, 229, 232 

ignorance, 518 

induction, 107-108 
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model-building, 611, 614 

parallel constraint satisfaction, 491 


854 


Model(s) (cont.) 
perfect market model, 617-619 
population model, 556, 561 
preferential, 341 
rational, 453 
of rationality, 507, 512, 747-751 
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empirical vs. genuine, 18, 25, 439, 440 
methodology of, 20-21 


Subject Index 


Normativity, 44-45, 129, 368-369, 373. See also Normative 
theory/theorizing 
Noiis, 71, 72-74. See also Logos 
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contrary-to-duty, 648 
Occam’s razor, 45 
Operation, formal, 90, 782 
Operational parsimony, 730 
Opinion pooling, 582 
Optimization, 507 
under constraints, 508-509 
Organization, 618, 619 
of science, 751, 752 
Other-regarding preference, 616, 617. See also Altruism 
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Predictive ability, 488, 489 
Preface paradox, 208, 338 
Preference(s), 15, 421-422. See also Desire 
aggregation, 582, 595-596 
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